-- Logs begin at Tue 2022-06-07 13:36:34 UTC, end at Tue 2022-06-07 15:03:52 UTC. -- Jun 07 14:18:25.671755 dv-cinder-844974-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 07 14:18:25.681874 dv-cinder-844974-1-iscsi devstack@keystone.service[80374]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 07 14:18:25.687759 dv-cinder-844974-1-iscsi devstack@keystone.service[80374]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Jun 7 14:18:25 2022] *** Jun 07 14:18:25.687759 dv-cinder-844974-1-iscsi devstack@keystone.service[80374]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 07 14:18:25.687759 dv-cinder-844974-1-iscsi devstack@keystone.service[80374]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 07 14:18:25.687759 dv-cinder-844974-1-iscsi devstack@keystone.service[80374]: nodename: dv-cinder-844974-1-iscsi Jun 07 14:18:25.687759 dv-cinder-844974-1-iscsi devstack@keystone.service[80374]: machine: x86_64 Jun 07 14:18:25.688154 dv-cinder-844974-1-iscsi devstack@keystone.service[80374]: clock source: unix Jun 07 14:18:25.688154 dv-cinder-844974-1-iscsi devstack@keystone.service[80374]: pcre jit disabled Jun 07 14:18:25.688154 dv-cinder-844974-1-iscsi devstack@keystone.service[80374]: detected number of CPU cores: 4 Jun 07 14:18:25.688154 dv-cinder-844974-1-iscsi devstack@keystone.service[80374]: current working directory: / Jun 07 14:18:25.688154 dv-cinder-844974-1-iscsi devstack@keystone.service[80374]: detected binary path: /usr/bin/uwsgi-core Jun 07 14:18:25.688154 dv-cinder-844974-1-iscsi devstack@keystone.service[80374]: your processes number limit is 47680 Jun 07 14:18:25.688154 dv-cinder-844974-1-iscsi devstack@keystone.service[80374]: your memory page size is 4096 bytes Jun 07 14:18:25.688154 dv-cinder-844974-1-iscsi devstack@keystone.service[80374]: detected max file descriptor number: 2048 Jun 07 14:18:25.688154 dv-cinder-844974-1-iscsi devstack@keystone.service[80374]: lock engine: pthread robust mutexes Jun 07 14:18:25.688154 dv-cinder-844974-1-iscsi devstack@keystone.service[80374]: thunder lock: enabled Jun 07 14:18:25.688715 dv-cinder-844974-1-iscsi devstack@keystone.service[80374]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 07 14:18:25.688715 dv-cinder-844974-1-iscsi devstack@keystone.service[80374]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 07 14:18:25.737032 dv-cinder-844974-1-iscsi devstack@keystone.service[80374]: Python main interpreter initialized at 0x5603656ac9c0 Jun 07 14:18:25.737032 dv-cinder-844974-1-iscsi devstack@keystone.service[80374]: python threads support enabled Jun 07 14:18:25.737032 dv-cinder-844974-1-iscsi devstack@keystone.service[80374]: your server socket listen backlog is limited to 100 connections Jun 07 14:18:25.737032 dv-cinder-844974-1-iscsi devstack@keystone.service[80374]: your mercy for graceful operations on workers is 90 seconds Jun 07 14:18:25.737644 dv-cinder-844974-1-iscsi devstack@keystone.service[80374]: mapped 403077 bytes (393 KB) for 2 cores Jun 07 14:18:25.737644 dv-cinder-844974-1-iscsi devstack@keystone.service[80374]: *** Operational MODE: preforking *** Jun 07 14:18:25.737954 dv-cinder-844974-1-iscsi devstack@keystone.service[80374]: *** uWSGI is running in multiple interpreter mode *** Jun 07 14:18:25.737954 dv-cinder-844974-1-iscsi devstack@keystone.service[80374]: spawned uWSGI master process (pid: 80374) Jun 07 14:18:25.737921 dv-cinder-844974-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 07 14:18:25.739064 dv-cinder-844974-1-iscsi devstack@keystone.service[80374]: spawned uWSGI worker 1 (pid: 80387, cores: 1) Jun 07 14:18:25.739064 dv-cinder-844974-1-iscsi devstack@keystone.service[80374]: spawned uWSGI worker 2 (pid: 80388, cores: 1) Jun 07 14:18:25.739064 dv-cinder-844974-1-iscsi devstack@keystone.service[80374]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 07 14:18:31.540361 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 07 14:18:31.540361 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: from cryptography.utils import int_from_bytes Jun 07 14:18:31.541190 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 07 14:18:31.541190 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: from cryptography.utils import int_from_bytes Jun 07 14:18:31.541190 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 07 14:18:31.541190 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: from cryptography.utils import int_from_bytes Jun 07 14:18:31.541801 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 07 14:18:31.541801 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: from cryptography.utils import int_from_bytes Jun 07 14:18:31.591082 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 07 14:18:31.591264 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 07 14:18:31.591264 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 07 14:18:31.591428 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 07 14:18:31.591500 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] command line args: [] {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 07 14:18:31.591566 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] command line args: [] {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 07 14:18:31.591718 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 07 14:18:31.591805 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 07 14:18:31.591876 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] ================================================================================ {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 07 14:18:31.591937 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ================================================================================ {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 07 14:18:31.592140 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] admin_token = **** {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.592235 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] admin_token = **** {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.592407 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] config_dir = [] {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.592503 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] config_dir = [] {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.592587 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.592673 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.592774 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] config_source = [] {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.592853 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] config_source = [] {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.592930 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] debug = True {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.593008 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] debug = True {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.593128 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.593379 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.593519 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.593594 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.593663 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.593726 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.593914 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.594030 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.594167 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.594284 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.594392 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] list_limit = None {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.594487 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] list_limit = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.594580 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] log_config_append = None {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.594679 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] log_config_append = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.594786 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.594880 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.594984 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] log_dir = None {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.595085 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] log_dir = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.595191 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] log_file = None {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.595285 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] log_file = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.595388 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.595474 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.595599 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.595675 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.595789 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.595908 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.595988 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.596160 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.596238 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.596354 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.596493 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.596618 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.596696 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.596811 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.596890 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.597006 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.597087 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.597159 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.597284 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.597365 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.597520 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.597599 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.597755 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.597836 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.598007 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.598102 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.598260 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.598346 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.598538 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.598625 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.599042 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.85/identity {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.599146 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.85/identity {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.599221 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] publish_errors = False {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.599295 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] publish_errors = False {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.599446 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.599595 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.599677 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.599795 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.599878 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.599956 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.600072 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.600237 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.600313 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.600411 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.600562 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] standard_threads = False {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.600640 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] standard_threads = False {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.600798 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.600884 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.601046 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.601134 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.601213 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.601290 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.601391 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] use_journal = False {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.601481 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] use_journal = False {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.601555 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] use_json = False {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.601635 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] use_json = False {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.601730 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] use_stderr = False {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.601814 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] use_stderr = False {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.601892 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] use_syslog = False {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.601962 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] use_syslog = False {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.602128 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.602219 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 14:18:31.602386 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.602465 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.602629 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.602711 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.602891 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.602977 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.603134 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.603212 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.603427 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.603541 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.603663 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.603738 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.603892 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.603972 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.604132 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.604208 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.604395 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.604486 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.604654 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.604739 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.604908 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.604987 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.605148 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.605226 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.605393 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.605484 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.605632 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.605720 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.605884 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.605968 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.606136 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.606219 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.606374 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.606457 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.606632 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.606727 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.606893 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.607009 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.607137 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.607222 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.607374 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.607458 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.607634 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.607710 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.607902 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.607997 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.608156 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.608236 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.608427 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.608519 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.608668 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.608745 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.608902 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.608981 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.609137 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.609222 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.609376 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.609452 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.609626 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.609731 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.609881 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.609984 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.610113 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.610250 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.610356 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.610452 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.610586 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.610669 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.610838 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.610924 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.611076 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.611154 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.611327 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.611405 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.611563 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.611653 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.611799 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.611883 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.612037 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.612116 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.612270 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.612355 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.612530 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.612617 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.612769 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.612847 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.613004 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.613217 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.613296 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.613467 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.613548 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.613711 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.613790 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.613950 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.614033 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.614184 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.614268 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.614419 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.614504 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.614659 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.614736 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.614898 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.614983 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.615138 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.615219 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.615381 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.615468 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.615622 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.615701 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.615864 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.615942 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.616105 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.616192 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.616347 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.616445 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.616606 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.616696 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.616847 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.616934 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.617324 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.617410 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.617571 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.617661 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.617890 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.618042 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.618126 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.618204 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.618384 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.618465 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.619076 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.619157 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.619333 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.619414 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.619652 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.619737 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.619899 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.619977 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.620161 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.620240 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.620450 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.620533 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.620706 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.620796 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.620984 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.621075 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.621229 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.621308 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.621501 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.621597 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.621768 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.621860 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.622015 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.622209 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.622290 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.622496 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.622579 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.622738 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.622822 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.622993 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.623112 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.623240 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.623318 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.623465 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.623546 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.623707 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.623786 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.623946 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.624029 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.624185 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.624265 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.624474 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.624555 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.624720 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.624799 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.624964 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.625044 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.625204 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.625282 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.625446 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.625525 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.625682 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.625762 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.625917 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.625995 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.626154 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.626239 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.626386 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.626465 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.626621 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.626710 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.626856 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.626936 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.627097 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.627178 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.627332 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.627409 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.627574 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.627649 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.627812 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.627893 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.628062 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.628141 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.628309 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.628474 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.628582 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.628706 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.628822 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] auth.external = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.628905 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.629060 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.629129 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.629301 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] auth.external = None {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.629380 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.629536 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.629640 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.629822 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] auth.password = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.629902 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.630058 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] auth.token = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.630136 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.630290 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] auth.password = None {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.630370 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.630524 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] auth.token = None {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.630607 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.630764 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.630851 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.631000 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.631079 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.631248 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.631325 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.631483 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.631641 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.631723 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.631802 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.631949 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] credential.caching = True {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.632029 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.632196 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.632285 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.632460 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] credential.caching = True {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.632543 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.632715 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.632794 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.632955 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.633049 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.633207 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.633335 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.633443 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.633522 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.633689 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.633782 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.633927 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.634008 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.634193 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.634271 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.634449 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.634539 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.635224 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.635311 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.635486 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.635565 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.635773 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.635859 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.636022 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.636106 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.636287 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.636413 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.636557 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.636649 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] federation.caching = True {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.636801 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.636880 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.637048 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] federation.caching = True {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.637125 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.637290 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.637378 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.637540 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.637618 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.637790 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.637867 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.638027 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.638116 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.638279 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.638364 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.638524 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.638602 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.638780 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.638867 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.639021 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.639099 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.639273 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.639352 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.639511 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.639596 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] identity.caching = True {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.639761 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.639840 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.640000 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] identity.caching = True {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.640085 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.640241 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.640328 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.640508 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.640600 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.640748 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.640989 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.642373 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.642634 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.642835 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.642916 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.643106 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.643185 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.643368 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.643449 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.643663 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.643741 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.643912 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.643992 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.644160 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.644249 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.644443 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.644538 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.644711 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.644792 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.644974 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.645055 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.645235 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.645313 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.645495 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.645574 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.645763 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.645843 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.646015 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.646093 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.646262 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.646341 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.646504 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.646586 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.646753 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.646834 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.646992 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.647069 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.647239 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.647331 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.647472 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.647552 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.647718 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.647797 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.647966 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.648047 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.648204 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.648283 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.648483 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.648565 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.648724 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.648805 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.648976 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.649054 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.649227 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.649305 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.649469 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.649552 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.649704 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.649782 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.649949 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.650029 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.650223 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.650354 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.650522 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.650673 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.650779 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.650966 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.651058 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.651216 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.651295 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.651968 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.652055 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.652222 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.652311 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.652491 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.652573 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.652739 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.652818 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.653004 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.653097 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.653261 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.653350 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.653501 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.653581 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.653739 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.653827 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.653985 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.654064 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.654230 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.654310 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.654470 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.user = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.654601 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.654711 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.654791 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.654953 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.655033 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.655192 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] ldap.user = None {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.655282 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.655437 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.655517 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.655690 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.655769 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.655930 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.656018 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.656185 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.656266 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.656464 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.656547 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.656711 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.656794 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.656953 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.657032 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.657191 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.657270 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.657427 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.657507 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.657665 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.657752 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.657906 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.657988 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.658135 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.658211 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.658382 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.658461 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.658634 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.658714 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.658879 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.658960 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.659128 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.659211 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.659371 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.659449 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.659606 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.659685 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.659845 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.659923 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.660084 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.660162 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.660324 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.660420 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.660590 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.660669 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.660830 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.660909 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.661083 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.661176 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.661321 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.661401 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.661561 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.661639 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.661806 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.661887 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.662042 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.662121 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.662283 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.662361 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.662524 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.662604 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.662762 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.662841 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.662998 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.663077 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.663241 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.663322 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.663484 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.663563 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] resource.caching = True {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.663720 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.663800 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.663961 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.664042 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.664204 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] resource.caching = True {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.664283 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.664483 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.664567 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.664737 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.664818 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.664977 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.665056 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.665229 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.665306 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.665470 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.665551 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.665709 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.665787 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.665974 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] role.caching = True {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.666096 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.666233 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] role.driver = sql {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.666316 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.666549 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.666641 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.666717 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] role.caching = True {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.666777 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.666956 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] role.driver = sql {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.667038 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.667190 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.667286 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.667445 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.667525 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.667680 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.667762 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.667934 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.668013 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.668191 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.668311 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.668458 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.668542 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.668715 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.668794 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.668965 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.669053 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.669217 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.669296 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.669451 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.669536 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.669703 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.669791 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.669954 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.670138 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.670218 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.670381 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.670460 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.670641 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.670860 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.670941 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.671176 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.671276 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.671352 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.671432 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.671598 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.671678 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.671851 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.671930 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.672104 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.672195 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.672343 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.672481 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.672614 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.672693 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.672856 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.672950 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.673095 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.673176 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.673357 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.673439 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.673602 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.673746 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.673828 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.673900 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.674079 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.674160 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.674318 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.674405 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.674556 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.674635 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.674798 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] token.caching = True {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.674877 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.675037 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.675124 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.675271 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.675351 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.675511 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] token.caching = True {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.675590 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.675751 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.675833 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.676005 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.676084 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.676245 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.676324 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.676518 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.676634 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.676772 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.676856 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.677016 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.677095 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.677258 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.677339 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.677504 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.677584 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.677745 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.677825 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.677985 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.678065 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.678228 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.678308 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.678467 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.678546 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.678714 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.678795 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.678965 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.679044 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.679199 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.679278 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] database.connection = **** {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.679444 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.679533 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.679744 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.679826 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.679986 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.680109 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] database.connection = **** {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.680234 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.680324 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.680498 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.680594 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.680739 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.680818 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.680978 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.681057 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.681217 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.681303 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.681458 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.681565 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.681743 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.681823 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.681987 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.682075 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.682227 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.682305 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.682466 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.682544 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.682710 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.682797 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.682947 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.683026 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.683186 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.683265 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.683429 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.683516 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.683669 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.683747 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.683843 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 07 14:18:31.683923 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.684131 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.684383 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 14:18:31.684564 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80387) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 07 14:18:31.713950 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:18:31.714191 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:18:31.714286 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:18:31.714423 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:18:31.714514 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:18:31.714666 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:18:31.723091 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:18:31.723204 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:18:31.751897 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:18:31.752118 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:18:31.752207 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:18:31.752284 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:18:31.806937 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:18:31.807109 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:18:31.856946 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:18:31.857199 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:18:31.857396 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:18:31.857534 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:18:31.925782 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:18:31.926005 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:18:31.926094 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:18:31.926225 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:18:31.926316 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:18:31.926500 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:18:31.926608 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:18:31.926688 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:18:31.926751 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:18:31.926899 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:18:31.926996 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:18:31.927149 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:18:31.951631 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:18:31.951821 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:18:31.951895 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:18:31.952047 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:18:31.952137 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:18:31.952282 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:18:31.952385 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:18:31.952528 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:18:31.952665 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:18:31.952759 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:18:31.952851 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:18:31.952929 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:18:31.953004 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:18:31.953173 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:18:31.964644 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:18:31.964888 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:18:31.977682 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:18:31.977970 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:18:31.978062 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:18:31.978132 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:18:31.978215 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:18:31.978480 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:18:31.978565 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:18:31.978629 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:18:31.978689 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:18:31.978932 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:18:31.979043 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:18:31.979157 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:18:31.979292 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:18:31.979390 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80387) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 14:18:31.997404 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:31.997635 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 07 14:18:31.997932 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:31.998058 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:31.998186 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 07 14:18:31.998313 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 07 14:18:31.998454 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:31.998576 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:31.998701 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:31.998869 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 07 14:18:31.998995 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:31.999125 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:31.999260 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:31.999380 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:31.999380 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:31.999648 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.010771 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.010976 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.011243 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.011329 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.011522 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.011604 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.011792 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.012246 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.019946 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80388) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 14:18:32.020496 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80387) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 14:18:32.023286 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80388) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 14:18:32.023565 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.029479 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80387) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 14:18:32.029570 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.029640 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.029713 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.029775 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.029849 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.029912 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.029973 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.030032 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.030091 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.030165 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.030239 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.030312 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.030383 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.030455 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.030523 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.030602 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.030677 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.030749 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.030749 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.054102 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.054642 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.055600 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80388) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 14:18:32.055864 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.056246 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80387) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 14:18:32.056585 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.063450 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80388) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 14:18:32.063450 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.063450 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.063686 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80387) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 14:18:32.063686 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.063686 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.069393 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80388) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 14:18:32.069679 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.069948 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80387) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 14:18:32.070238 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.074243 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80388) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 14:18:32.074537 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.074804 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80387) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 14:18:32.074937 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.075094 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.075178 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.075372 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.075456 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.075725 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.075833 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.075931 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.076003 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.076221 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.076304 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.076519 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.076903 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.095539 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.095816 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.096362 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.096639 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.100476 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80388) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 14:18:32.101296 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80387) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 14:18:32.104829 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.105130 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.105776 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.106075 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.110229 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80388) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 14:18:32.111181 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80387) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 14:18:32.113881 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80388) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 14:18:32.114840 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80387) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 14:18:32.117580 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.117886 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.118175 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.118464 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.118800 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.119014 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.119108 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.119342 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.119455 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.119692 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.119981 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.120272 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.138760 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80388) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 14:18:32.139056 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.139340 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.139602 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.139827 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80387) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 14:18:32.140127 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.140426 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.140694 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.147628 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.148730 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.151592 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.151885 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.152643 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.152920 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.154993 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80388) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 14:18:32.155281 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.155598 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.155862 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.155990 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80387) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 14:18:32.156149 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.156274 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.156593 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.156879 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.157169 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.170219 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80388) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 14:18:32.170509 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.171052 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.171052 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.171248 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80387) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 14:18:32.171358 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.171540 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.171634 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.171827 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.171911 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.172112 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.172444 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.172689 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.172976 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.191162 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80387) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 14:18:32.193546 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80388) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 14:18:32.194499 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80387) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 14:18:32.196985 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80388) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 14:18:32.200960 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.200960 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.201118 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.201399 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80387) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 14:18:32.201692 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.201977 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.202169 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.203846 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80388) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 14:18:32.204134 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.204464 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.209442 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.211076 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80387) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 14:18:32.211825 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.214061 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80388) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 14:18:32.215079 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.215370 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.215654 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.215930 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.217480 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.217774 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.218065 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.218352 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.225172 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80387) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 14:18:32.225424 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.225703 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.227508 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80388) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 14:18:32.227801 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.228084 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.234303 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80387) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 14:18:32.234583 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.234865 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.235146 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.235419 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.235695 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.235970 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.236244 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.236543 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.236766 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80388) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 14:18:32.236849 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.237049 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.237130 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.237332 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.237499 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.237610 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.237691 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.237869 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80387) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.237948 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.238203 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.238437 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.238717 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.238991 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.239272 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.239539 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.239820 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.240095 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.240357 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 14:18:32.277252 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x5603656ac9c0 pid: 80387 (default app) Jun 07 14:18:32.279611 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x5603656ac9c0 pid: 80388 (default app) Jun 07 14:18:32.280725 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1fc8a4-798d-4072-a973-2e2e7a0c6ba2 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:18:32.281052 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1fc8a4-798d-4072-a973-2e2e7a0c6ba2 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:18:32.281332 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1fc8a4-798d-4072-a973-2e2e7a0c6ba2 None None] PATH_INFO: `/v3` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:18:32.285488 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1/1] 10.222.0.85 () {50 vars in 791 bytes} [Tue Jun 7 14:18:32 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:18:41.745724 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563e4ea3-bd5e-4fa1-af1f-0261f7f24955 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:18:41.746294 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563e4ea3-bd5e-4fa1-af1f-0261f7f24955 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:18:41.746294 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563e4ea3-bd5e-4fa1-af1f-0261f7f24955 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:18:41.750552 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1/2] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:18:41 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 14:18:41.756571 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40632a77-32fd-4aa6-8a7d-54eda0de25b0 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:18:41.757011 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40632a77-32fd-4aa6-8a7d-54eda0de25b0 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:18:41.757112 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40632a77-32fd-4aa6-8a7d-54eda0de25b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:18:41.836214 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: INFO dbcounter [None req-40632a77-32fd-4aa6-8a7d-54eda0de25b0 None None] Registered counter for database keystone Jun 07 14:18:41.859319 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG oslo_db.sqlalchemy.engines [None req-40632a77-32fd-4aa6-8a7d-54eda0de25b0 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80387) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 07 14:18:41.865898 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writer thread running {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 07 14:18:42.162504 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG passlib.handlers.bcrypt [None req-40632a77-32fd-4aa6-8a7d-54eda0de25b0 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80387) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 07 14:18:42.162830 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG passlib.handlers.bcrypt [None req-40632a77-32fd-4aa6-8a7d-54eda0de25b0 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80387) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 07 14:18:42.220255 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-40632a77-32fd-4aa6-8a7d-54eda0de25b0 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:18:42.221977 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-40632a77-32fd-4aa6-8a7d-54eda0de25b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:18:42.356820 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 2/3] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:18:41 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 601 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 07 14:18:42.531240 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563e4ea3-bd5e-4fa1-af1f-0261f7f24955 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:18:42.600308 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: INFO dbcounter [None req-563e4ea3-bd5e-4fa1-af1f-0261f7f24955 None None] Registered counter for database keystone Jun 07 14:18:42.622177 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG oslo_db.sqlalchemy.engines [None req-563e4ea3-bd5e-4fa1-af1f-0261f7f24955 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80388) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 07 14:18:42.629549 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writer thread running {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 07 14:18:42.862779 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563e4ea3-bd5e-4fa1-af1f-0261f7f24955 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:18:42.867693 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692c0aa7-a6f4-40db-840b-aa059161fb37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:18:42.868481 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692c0aa7-a6f4-40db-840b-aa059161fb37 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:18:42.869010 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692c0aa7-a6f4-40db-840b-aa059161fb37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:18:42.869847 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692c0aa7-a6f4-40db-840b-aa059161fb37 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:18:42.879142 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692c0aa7-a6f4-40db-840b-aa059161fb37 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:18:42.930875 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: WARNING keystone.common.rbac_enforcer.enforcer [None req-692c0aa7-a6f4-40db-840b-aa059161fb37 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 07 14:18:42.931227 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692c0aa7-a6f4-40db-840b-aa059161fb37 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:18:42.942436 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 2/4] 10.222.0.85 () {58 vars in 1324 bytes} [Tue Jun 7 14:18:42 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 07 14:18:42.949468 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40632a77-32fd-4aa6-8a7d-54eda0de25b0 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:18:42.968254 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40632a77-32fd-4aa6-8a7d-54eda0de25b0 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:18:42.972856 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23bb6f1-6b47-4866-8677-6a506edf9a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:18:42.974011 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23bb6f1-6b47-4866-8677-6a506edf9a25 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:18:42.974298 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23bb6f1-6b47-4866-8677-6a506edf9a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:18:42.974586 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23bb6f1-6b47-4866-8677-6a506edf9a25 admin admin] PATH_INFO: `/v3/projects/61703df7c02543d6ba1f949d9811ee43` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:18:42.976143 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23bb6f1-6b47-4866-8677-6a506edf9a25 admin admin] RBAC: Authorizing `identity:get_project(project_id=61703df7c02543d6ba1f949d9811ee43)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:18:43.040088 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: WARNING keystone.common.rbac_enforcer.enforcer [None req-a23bb6f1-6b47-4866-8677-6a506edf9a25 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 07 14:18:43.040457 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23bb6f1-6b47-4866-8677-6a506edf9a25 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:18:43.050655 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 3/5] 10.222.0.85 () {56 vars in 1207 bytes} [Tue Jun 7 14:18:42 2022] GET /identity/v3/projects/61703df7c02543d6ba1f949d9811ee43 => generated 344 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:18:45.320619 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e3c938-ed7f-491b-b026-943578d3136a None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:18:45.321162 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e3c938-ed7f-491b-b026-943578d3136a None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:18:45.321162 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e3c938-ed7f-491b-b026-943578d3136a None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:18:45.325337 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 3/6] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:18:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 14:18:45.332266 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10061dd2-f6f1-45f2-bffa-150852c26410 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:18:45.332834 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10061dd2-f6f1-45f2-bffa-150852c26410 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:18:45.332921 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10061dd2-f6f1-45f2-bffa-150852c26410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:18:45.354041 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-10061dd2-f6f1-45f2-bffa-150852c26410 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:18:45.355643 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-10061dd2-f6f1-45f2-bffa-150852c26410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:18:45.363817 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 4/7] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:18:45 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 07 14:18:45.388089 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e3c938-ed7f-491b-b026-943578d3136a None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:18:45.407329 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e3c938-ed7f-491b-b026-943578d3136a None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:18:45.411658 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5952b7-259c-4a35-9ddb-ec5c59527327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:18:45.412319 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5952b7-259c-4a35-9ddb-ec5c59527327 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:18:45.412632 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5952b7-259c-4a35-9ddb-ec5c59527327 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:18:45.412914 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5952b7-259c-4a35-9ddb-ec5c59527327 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:18:45.422043 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5952b7-259c-4a35-9ddb-ec5c59527327 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:18:45.423832 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5952b7-259c-4a35-9ddb-ec5c59527327 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:18:45.438112 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 4/8] 10.222.0.85 () {58 vars in 1324 bytes} [Tue Jun 7 14:18:45 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 14:18:45.444653 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10061dd2-f6f1-45f2-bffa-150852c26410 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:18:45.456487 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10061dd2-f6f1-45f2-bffa-150852c26410 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:18:45.461089 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf8241a-32e4-494f-9211-d226371f9fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:18:45.461972 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf8241a-32e4-494f-9211-d226371f9fd3 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:18:45.462258 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf8241a-32e4-494f-9211-d226371f9fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:18:45.463065 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf8241a-32e4-494f-9211-d226371f9fd3 admin admin] PATH_INFO: `/v3/users/3bd6d2364d1d4e71977a1f04d01cd108` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:18:45.464706 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf8241a-32e4-494f-9211-d226371f9fd3 admin admin] RBAC: Authorizing `identity:get_user(user_id=3bd6d2364d1d4e71977a1f04d01cd108)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:18:45.466546 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf8241a-32e4-494f-9211-d226371f9fd3 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:18:45.468163 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 5/9] 10.222.0.85 () {56 vars in 1198 bytes} [Tue Jun 7 14:18:45 2022] GET /identity/v3/users/3bd6d2364d1d4e71977a1f04d01cd108 => generated 247 bytes in 25 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:18:47.912663 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565ad5bb-520b-4a52-9e56-8d4d148e5e6e None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:18:47.912663 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565ad5bb-520b-4a52-9e56-8d4d148e5e6e None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:18:47.913729 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565ad5bb-520b-4a52-9e56-8d4d148e5e6e None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:18:47.914474 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 5/10] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:18:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:18:47.921220 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71cb67d-a166-4985-91d4-54e3b6c185eb None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:18:47.921758 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71cb67d-a166-4985-91d4-54e3b6c185eb None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:18:47.921872 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71cb67d-a166-4985-91d4-54e3b6c185eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:18:47.929902 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d58902-5c40-4fc8-ab34-19b3648a0288 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:18:47.930304 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d58902-5c40-4fc8-ab34-19b3648a0288 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:18:47.930591 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d58902-5c40-4fc8-ab34-19b3648a0288 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:18:47.932023 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 6/11] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:18:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 14:18:47.937599 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431fa35a-9d72-4a1d-896b-deb98cd3c794 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:18:47.937918 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431fa35a-9d72-4a1d-896b-deb98cd3c794 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:18:47.938211 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431fa35a-9d72-4a1d-896b-deb98cd3c794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:18:47.939950 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-a71cb67d-a166-4985-91d4-54e3b6c185eb None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:18:47.941194 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-a71cb67d-a166-4985-91d4-54e3b6c185eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:18:47.983803 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG passlib.handlers.bcrypt [None req-431fa35a-9d72-4a1d-896b-deb98cd3c794 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80388) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 07 14:18:47.984232 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG passlib.handlers.bcrypt [None req-431fa35a-9d72-4a1d-896b-deb98cd3c794 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80388) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 07 14:18:48.031786 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-431fa35a-9d72-4a1d-896b-deb98cd3c794 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:18:48.033444 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-431fa35a-9d72-4a1d-896b-deb98cd3c794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:18:48.033971 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 6/12] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:18:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:18:48.058932 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71cb67d-a166-4985-91d4-54e3b6c185eb None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:18:48.091621 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:18:48.183418 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 7/13] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:18:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 14:18:48.206807 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71cb67d-a166-4985-91d4-54e3b6c185eb None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:18:48.207385 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431fa35a-9d72-4a1d-896b-deb98cd3c794 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:18:48.246927 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400e390e-d874-414d-9fe1-5cc63827a319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:18:48.248161 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400e390e-d874-414d-9fe1-5cc63827a319 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:18:48.248703 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400e390e-d874-414d-9fe1-5cc63827a319 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:18:48.249078 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400e390e-d874-414d-9fe1-5cc63827a319 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:18:48.264645 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400e390e-d874-414d-9fe1-5cc63827a319 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:18:48.278019 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-400e390e-d874-414d-9fe1-5cc63827a319 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:18:48.278019 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:18:48.278019 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:18:48.278019 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:18:48.278019 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:18:48.278019 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:18:48.278019 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:18:48.278019 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:18:48.278019 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:18:48.278019 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:18:48.278019 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:18:48.278019 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:18:48.278019 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:18:48.278019 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:18:48.278019 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:18:48.278019 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:18:48.279256 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:18:48.279256 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:18:48.279256 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:18:48.279256 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:18:48.279256 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:18:48.279256 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:18:48.279256 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:18:48.279256 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:18:48.279256 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:18:48.279256 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:18:48.279256 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:18:48.279256 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:18:48.279256 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:18:48.279256 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:18:48.279256 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:18:48.279256 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:18:48.285622 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:18:48.285622 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:18:48.285622 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:18:48.285622 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:18:48.285622 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:18:48.285622 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:18:48.285622 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:18:48.285622 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:18:48.285622 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:18:48.285622 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:18:48.285622 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:18:48.285622 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:18:48.285622 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 7/14] 10.222.0.85 () {56 vars in 1096 bytes} [Tue Jun 7 14:18:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:18:48.286248 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400e390e-d874-414d-9fe1-5cc63827a319 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:18:48.326788 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431fa35a-9d72-4a1d-896b-deb98cd3c794 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:18:48.329235 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:18:48.380390 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcc3e1a-8c2e-437c-be2b-9c45e5aba522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:18:48.381334 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcc3e1a-8c2e-437c-be2b-9c45e5aba522 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:18:48.381621 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcc3e1a-8c2e-437c-be2b-9c45e5aba522 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:18:48.381901 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcc3e1a-8c2e-437c-be2b-9c45e5aba522 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:18:48.386271 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:18:48.405351 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400e390e-d874-414d-9fe1-5cc63827a319 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:18:48.445492 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7155232-b6bc-43f1-ab5b-37997e07abbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:18:48.446221 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7155232-b6bc-43f1-ab5b-37997e07abbb None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:18:48.446673 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7155232-b6bc-43f1-ab5b-37997e07abbb None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:18:48.447587 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7155232-b6bc-43f1-ab5b-37997e07abbb None admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:18:48.448699 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7155232-b6bc-43f1-ab5b-37997e07abbb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:18:48.449192 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7155232-b6bc-43f1-ab5b-37997e07abbb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:18:48.450770 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7155232-b6bc-43f1-ab5b-37997e07abbb None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:18:48.464001 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 8/15] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jun 7 14:18:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:18:48.469745 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7155232-b6bc-43f1-ab5b-37997e07abbb None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:18:48.497486 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcc3e1a-8c2e-437c-be2b-9c45e5aba522 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:18:48.499054 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcc3e1a-8c2e-437c-be2b-9c45e5aba522 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:18:48.593017 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7155232-b6bc-43f1-ab5b-37997e07abbb None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:18:48.628471 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 8/16] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jun 7 14:18:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 07 14:18:48.633335 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d729773d-801d-4732-981c-ad3b8ed95200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:18:48.633335 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d729773d-801d-4732-981c-ad3b8ed95200 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:18:48.633335 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d729773d-801d-4732-981c-ad3b8ed95200 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:18:48.633335 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d729773d-801d-4732-981c-ad3b8ed95200 None admin] PATH_INFO: `/v3/users/3bd6d2364d1d4e71977a1f04d01cd108` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:18:48.633715 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d729773d-801d-4732-981c-ad3b8ed95200 None admin] RBAC: Authorizing `identity:get_user(user_id=3bd6d2364d1d4e71977a1f04d01cd108)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:18:48.635038 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcc3e1a-8c2e-437c-be2b-9c45e5aba522 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:18:48.635629 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d729773d-801d-4732-981c-ad3b8ed95200 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:18:48.637375 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 9/17] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jun 7 14:18:48 2022] GET /identity/v3/users/3bd6d2364d1d4e71977a1f04d01cd108 => generated 247 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:18:48.639207 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:18:48.643103 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d729773d-801d-4732-981c-ad3b8ed95200 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:18:48.659707 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:18:48.765878 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcc3e1a-8c2e-437c-be2b-9c45e5aba522 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:18:48.766166 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d729773d-801d-4732-981c-ad3b8ed95200 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:18:48.817997 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617d28c0-f43b-4372-851f-74737f13e9db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:18:48.818953 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617d28c0-f43b-4372-851f-74737f13e9db None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:18:48.819345 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617d28c0-f43b-4372-851f-74737f13e9db None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:18:48.819696 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617d28c0-f43b-4372-851f-74737f13e9db None admin] PATH_INFO: `/v3/domains/default` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:18:48.820695 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3447cdcd-bc3c-4c75-b550-fdcc7bfb8504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:18:48.821089 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617d28c0-f43b-4372-851f-74737f13e9db None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:18:48.821671 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3447cdcd-bc3c-4c75-b550-fdcc7bfb8504 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:18:48.822065 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3447cdcd-bc3c-4c75-b550-fdcc7bfb8504 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:18:48.822429 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3447cdcd-bc3c-4c75-b550-fdcc7bfb8504 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:18:48.822820 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617d28c0-f43b-4372-851f-74737f13e9db None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:18:48.824661 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 10/18] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:18:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:18:48.832155 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617d28c0-f43b-4372-851f-74737f13e9db None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:18:48.843618 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3447cdcd-bc3c-4c75-b550-fdcc7bfb8504 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:18:48.845295 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3447cdcd-bc3c-4c75-b550-fdcc7bfb8504 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:18:48.873034 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-3447cdcd-bc3c-4c75-b550-fdcc7bfb8504 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:18:48.873034 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:18:48.873034 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:18:48.873034 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:18:48.873034 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:18:48.873034 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:18:48.873034 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:18:48.873034 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:18:48.873034 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:18:48.873034 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:18:48.873034 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:18:48.873034 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:18:48.873034 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:18:48.873034 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:18:48.873034 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:18:48.874039 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:18:48.874039 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:18:48.874039 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:18:48.874039 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:18:48.874039 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:18:48.874039 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:18:48.874039 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:18:48.874039 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:18:48.874039 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:18:48.874039 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:18:48.874039 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:18:48.874039 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:18:48.874039 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:18:48.874039 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:18:48.874039 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:18:48.874039 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:18:48.874039 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:18:48.875138 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:18:48.875138 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:18:48.875138 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:18:48.875138 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:18:48.875138 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 9/19] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:18:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:18:48.881336 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3447cdcd-bc3c-4c75-b550-fdcc7bfb8504 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:18:48.946433 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:18:48.947632 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:18:48.966896 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617d28c0-f43b-4372-851f-74737f13e9db None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:18:48.993214 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3447cdcd-bc3c-4c75-b550-fdcc7bfb8504 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:18:49.005626 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0600ba-536b-44e2-9ea7-7cf2e7b7679c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:18:49.006308 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0600ba-536b-44e2-9ea7-7cf2e7b7679c None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:18:49.006595 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0600ba-536b-44e2-9ea7-7cf2e7b7679c None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:18:49.006881 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0600ba-536b-44e2-9ea7-7cf2e7b7679c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:18:49.007733 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0600ba-536b-44e2-9ea7-7cf2e7b7679c None admin] RBAC: Adding query filter params (role.id=9baff2b18f2649f7b757ff2e1f63c853, scope.domain.id=default, user.id=3bd6d2364d1d4e71977a1f04d01cd108) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:18:49.008051 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0600ba-536b-44e2-9ea7-7cf2e7b7679c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:18:49.009664 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0600ba-536b-44e2-9ea7-7cf2e7b7679c None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:18:49.018783 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 11/20] 10.222.0.85 () {56 vars in 1322 bytes} [Tue Jun 7 14:18:48 2022] GET /identity/v3/role_assignments?user.id=3bd6d2364d1d4e71977a1f04d01cd108&scope.domain.id=default&role.id=9baff2b18f2649f7b757ff2e1f63c853 => generated 233 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:18:49.037584 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbc4430-a09e-4ec2-a87d-642265e6d6b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:18:49.038288 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbc4430-a09e-4ec2-a87d-642265e6d6b9 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:18:49.038579 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbc4430-a09e-4ec2-a87d-642265e6d6b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:18:49.038863 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbc4430-a09e-4ec2-a87d-642265e6d6b9 None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:18:49.039653 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbc4430-a09e-4ec2-a87d-642265e6d6b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:18:49.039966 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbc4430-a09e-4ec2-a87d-642265e6d6b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:18:49.042010 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbc4430-a09e-4ec2-a87d-642265e6d6b9 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:18:49.064332 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 10/21] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jun 7 14:18:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:18:51.702816 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9517c0-7690-46ab-9030-1616366dd26d None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:18:51.703873 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9517c0-7690-46ab-9030-1616366dd26d None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:18:51.704490 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9517c0-7690-46ab-9030-1616366dd26d None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:18:51.706217 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 12/22] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:18:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:18:51.714196 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2840f34-cf99-474e-bfcb-995fda018f3d None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:18:51.715187 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2840f34-cf99-474e-bfcb-995fda018f3d None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:18:51.715677 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2840f34-cf99-474e-bfcb-995fda018f3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:18:51.737679 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-c2840f34-cf99-474e-bfcb-995fda018f3d None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:18:51.739438 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-c2840f34-cf99-474e-bfcb-995fda018f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:18:51.848421 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 11/23] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:18:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:18:51.892714 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9517c0-7690-46ab-9030-1616366dd26d None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:18:52.065726 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9517c0-7690-46ab-9030-1616366dd26d None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:18:52.117766 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6459e743-418f-408a-92c1-3d5fa1c8feb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:18:52.119146 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6459e743-418f-408a-92c1-3d5fa1c8feb7 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:18:52.119456 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6459e743-418f-408a-92c1-3d5fa1c8feb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:18:52.119735 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6459e743-418f-408a-92c1-3d5fa1c8feb7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:18:52.135401 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6459e743-418f-408a-92c1-3d5fa1c8feb7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:18:52.143475 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-6459e743-418f-408a-92c1-3d5fa1c8feb7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:18:52.143475 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:18:52.143475 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:18:52.143475 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:18:52.143475 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:18:52.143475 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:18:52.143475 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:18:52.143475 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:18:52.143475 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:18:52.143475 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:18:52.143475 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:18:52.143475 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:18:52.143475 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:18:52.143475 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:18:52.143475 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:18:52.143475 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:18:52.144477 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:18:52.144477 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:18:52.144477 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:18:52.144477 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:18:52.144477 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:18:52.144477 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:18:52.144477 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:18:52.144477 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:18:52.144477 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:18:52.144477 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:18:52.144477 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:18:52.144477 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:18:52.144477 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:18:52.144477 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:18:52.144477 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:18:52.144477 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:18:52.145803 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:18:52.145803 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:18:52.145803 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:18:52.145803 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:18:52.145803 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:18:52.145803 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:18:52.145803 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:18:52.145803 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:18:52.145803 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:18:52.145803 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:18:52.145803 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:18:52.145803 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:18:52.145803 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 13/24] 10.222.0.85 () {56 vars in 1096 bytes} [Tue Jun 7 14:18:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:18:52.152140 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2840f34-cf99-474e-bfcb-995fda018f3d None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:18:52.234523 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:18:52.247141 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e593be-fede-430d-9daa-fad5107f0423 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:18:52.247439 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e593be-fede-430d-9daa-fad5107f0423 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:18:52.247901 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e593be-fede-430d-9daa-fad5107f0423 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:18:52.249787 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 14/25] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:18:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:18:52.256255 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec348d49-e2c7-4fa0-8c52-c6d9e0c3a4d8 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:18:52.257501 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec348d49-e2c7-4fa0-8c52-c6d9e0c3a4d8 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:18:52.258172 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec348d49-e2c7-4fa0-8c52-c6d9e0c3a4d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:18:52.273552 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:18:52.296464 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-ec348d49-e2c7-4fa0-8c52-c6d9e0c3a4d8 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:18:52.297715 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-ec348d49-e2c7-4fa0-8c52-c6d9e0c3a4d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:18:52.375488 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2840f34-cf99-474e-bfcb-995fda018f3d None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:18:52.457175 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b61d5f-93df-45d9-bcf6-5a90cd6b13c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:18:52.459741 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b61d5f-93df-45d9-bcf6-5a90cd6b13c6 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:18:52.460308 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b61d5f-93df-45d9-bcf6-5a90cd6b13c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:18:52.461590 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b61d5f-93df-45d9-bcf6-5a90cd6b13c6 None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:18:52.463151 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b61d5f-93df-45d9-bcf6-5a90cd6b13c6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:18:52.464283 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b61d5f-93df-45d9-bcf6-5a90cd6b13c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:18:52.466133 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b61d5f-93df-45d9-bcf6-5a90cd6b13c6 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:18:52.474515 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 15/26] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:18:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 14:18:52.476408 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c137677e-b383-41ad-a413-d7d0a4fb4117 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:18:52.484407 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c137677e-b383-41ad-a413-d7d0a4fb4117 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:18:52.485458 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c137677e-b383-41ad-a413-d7d0a4fb4117 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:18:52.488271 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 16/27] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:18:52 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:18:52.495373 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee2f194-71e4-4b71-8620-fc81d37862a7 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:18:52.496006 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee2f194-71e4-4b71-8620-fc81d37862a7 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:18:52.496524 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee2f194-71e4-4b71-8620-fc81d37862a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:18:52.502554 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 12/28] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jun 7 14:18:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:18:52.511133 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b61d5f-93df-45d9-bcf6-5a90cd6b13c6 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:18:52.534997 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-4ee2f194-71e4-4b71-8620-fc81d37862a7 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:18:52.537390 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-4ee2f194-71e4-4b71-8620-fc81d37862a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:18:52.694115 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b61d5f-93df-45d9-bcf6-5a90cd6b13c6 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:18:52.712420 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 17/29] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:18:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 14:18:52.713999 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee2f194-71e4-4b71-8620-fc81d37862a7 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:18:52.724798 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:18:52.762696 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14803401-06d4-4b05-a29b-655e0225f0f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:18:52.764601 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14803401-06d4-4b05-a29b-655e0225f0f7 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:18:52.764911 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14803401-06d4-4b05-a29b-655e0225f0f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:18:52.765197 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14803401-06d4-4b05-a29b-655e0225f0f7 None admin] PATH_INFO: `/v3/users/3bd6d2364d1d4e71977a1f04d01cd108` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:18:52.769530 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14803401-06d4-4b05-a29b-655e0225f0f7 None admin] RBAC: Authorizing `identity:get_user(user_id=3bd6d2364d1d4e71977a1f04d01cd108)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:18:52.774010 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14803401-06d4-4b05-a29b-655e0225f0f7 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:18:52.780334 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 13/30] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jun 7 14:18:52 2022] GET /identity/v3/users/3bd6d2364d1d4e71977a1f04d01cd108 => generated 247 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:18:52.781316 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14803401-06d4-4b05-a29b-655e0225f0f7 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:18:52.893442 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:18:52.953409 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14803401-06d4-4b05-a29b-655e0225f0f7 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:18:52.986163 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee2f194-71e4-4b71-8620-fc81d37862a7 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:18:53.038163 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6a990d-7bc9-4703-ad5b-c4624302bcc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:18:53.039023 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6a990d-7bc9-4703-ad5b-c4624302bcc0 None admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:18:53.039477 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6a990d-7bc9-4703-ad5b-c4624302bcc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:18:53.039911 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6a990d-7bc9-4703-ad5b-c4624302bcc0 None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:18:53.042185 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6a990d-7bc9-4703-ad5b-c4624302bcc0 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:18:53.043276 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6a990d-7bc9-4703-ad5b-c4624302bcc0 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:18:53.070984 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24df72bc-b66d-4fff-83e9-b4a2aff1e851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:18:53.072415 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24df72bc-b66d-4fff-83e9-b4a2aff1e851 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:18:53.072718 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24df72bc-b66d-4fff-83e9-b4a2aff1e851 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:18:53.073083 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24df72bc-b66d-4fff-83e9-b4a2aff1e851 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:18:53.085652 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24df72bc-b66d-4fff-83e9-b4a2aff1e851 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:18:53.088646 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24df72bc-b66d-4fff-83e9-b4a2aff1e851 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:18:53.112588 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-24df72bc-b66d-4fff-83e9-b4a2aff1e851 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:18:53.112588 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:18:53.112588 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:18:53.112588 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:18:53.112588 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:18:53.112588 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:18:53.112588 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:18:53.112588 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:18:53.112588 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:18:53.112588 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:18:53.112588 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:18:53.112588 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:18:53.112588 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:18:53.112588 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:18:53.112588 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:18:53.113891 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:18:53.113891 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:18:53.113891 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:18:53.113891 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:18:53.113891 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:18:53.113891 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:18:53.113891 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:18:53.113891 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:18:53.113891 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:18:53.113891 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:18:53.113891 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:18:53.113891 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:18:53.113891 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:18:53.113891 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:18:53.113891 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:18:53.113891 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:18:53.113891 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:18:53.114980 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:18:53.114980 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:18:53.114980 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:18:53.114980 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:18:53.114980 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 18/31] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:18:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 400 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:18:53.115256 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24df72bc-b66d-4fff-83e9-b4a2aff1e851 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:18:53.287748 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24df72bc-b66d-4fff-83e9-b4a2aff1e851 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:18:53.312099 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 14/32] 10.222.0.85 () {60 vars in 1131 bytes} [Tue Jun 7 14:18:52 2022] POST /identity/v3/roles => generated 219 bytes in 531 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:18:53.313695 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b44c39-02e4-4013-b64f-61f6fd1e433b None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:18:53.313987 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b44c39-02e4-4013-b64f-61f6fd1e433b None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:18:53.314281 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b44c39-02e4-4013-b64f-61f6fd1e433b None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:18:53.319931 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:18:53.320120 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 15/33] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:18:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:18:53.320120 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b44c39-02e4-4013-b64f-61f6fd1e433b None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:18:53.358224 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28071fc2-620d-4540-b127-48e34a8637c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:18:53.359791 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28071fc2-620d-4540-b127-48e34a8637c7 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:18:53.360561 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28071fc2-620d-4540-b127-48e34a8637c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:18:53.361326 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:18:53.361464 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28071fc2-620d-4540-b127-48e34a8637c7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:18:53.363965 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28071fc2-620d-4540-b127-48e34a8637c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:18:53.366229 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28071fc2-620d-4540-b127-48e34a8637c7 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:18:53.370396 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 19/34] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:18:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:18:53.377992 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68839ee-4e5b-4909-ad9f-d156f9ad3629 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:18:53.378248 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68839ee-4e5b-4909-ad9f-d156f9ad3629 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:18:53.378506 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68839ee-4e5b-4909-ad9f-d156f9ad3629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:18:53.407456 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-d68839ee-4e5b-4909-ad9f-d156f9ad3629 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:18:53.409743 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-d68839ee-4e5b-4909-ad9f-d156f9ad3629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:18:53.458182 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b44c39-02e4-4013-b64f-61f6fd1e433b None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:18:53.506068 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e71fd3-d75f-428b-b707-738f34aa8dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:18:53.507053 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e71fd3-d75f-428b-b707-738f34aa8dce None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:18:53.507491 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e71fd3-d75f-428b-b707-738f34aa8dce None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:18:53.507926 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e71fd3-d75f-428b-b707-738f34aa8dce None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:18:53.508750 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e71fd3-d75f-428b-b707-738f34aa8dce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:18:53.509074 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e71fd3-d75f-428b-b707-738f34aa8dce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:18:53.510705 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e71fd3-d75f-428b-b707-738f34aa8dce None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:18:53.543229 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 20/35] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:18:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:18:53.543229 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68839ee-4e5b-4909-ad9f-d156f9ad3629 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:18:53.543505 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 16/36] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jun 7 14:18:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:18:53.544828 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff17a6d-839b-4201-8fc0-0968dbd9781f None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:18:53.545268 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff17a6d-839b-4201-8fc0-0968dbd9781f None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:18:53.545714 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff17a6d-839b-4201-8fc0-0968dbd9781f None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:18:53.547973 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 17/37] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:18:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:18:53.549655 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a887cd-6856-4b10-bb65-99b340607274 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:18:53.549978 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a887cd-6856-4b10-bb65-99b340607274 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:18:53.550266 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a887cd-6856-4b10-bb65-99b340607274 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:18:53.552662 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 18/38] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:18:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:18:53.553207 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a887cd-6856-4b10-bb65-99b340607274 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:18:53.697749 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:18:53.702887 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68839ee-4e5b-4909-ad9f-d156f9ad3629 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:18:53.706522 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a887cd-6856-4b10-bb65-99b340607274 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:18:53.720816 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:18:53.753017 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1888daf9-cc5e-43f3-8d2a-9a5a3ea2a74f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:18:53.753739 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b6ffcc-035a-4f41-8faf-e87d7946168b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:18:53.753951 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1888daf9-cc5e-43f3-8d2a-9a5a3ea2a74f None admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:18:53.754081 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1888daf9-cc5e-43f3-8d2a-9a5a3ea2a74f None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:18:53.754379 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1888daf9-cc5e-43f3-8d2a-9a5a3ea2a74f None admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:18:53.754832 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b6ffcc-035a-4f41-8faf-e87d7946168b None admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:18:53.755075 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1888daf9-cc5e-43f3-8d2a-9a5a3ea2a74f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:18:53.755221 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b6ffcc-035a-4f41-8faf-e87d7946168b None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:18:53.755455 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b6ffcc-035a-4f41-8faf-e87d7946168b None admin] PATH_INFO: `/v3/domains/default/users/3bd6d2364d1d4e71977a1f04d01cd108/roles/9baff2b18f2649f7b757ff2e1f63c853` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:18:53.756500 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1888daf9-cc5e-43f3-8d2a-9a5a3ea2a74f None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:18:53.757979 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b6ffcc-035a-4f41-8faf-e87d7946168b None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=3bd6d2364d1d4e71977a1f04d01cd108, role_id=9baff2b18f2649f7b757ff2e1f63c853)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:18:53.759573 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b6ffcc-035a-4f41-8faf-e87d7946168b None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:18:53.924745 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 19/39] 10.222.0.85 () {60 vars in 1141 bytes} [Tue Jun 7 14:18:53 2022] POST /identity/v3/projects => generated 301 bytes in 372 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:18:53.926051 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cca099-aa2d-4671-8d82-8de149455596 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:18:53.926350 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cca099-aa2d-4671-8d82-8de149455596 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:18:53.926636 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cca099-aa2d-4671-8d82-8de149455596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:18:53.948710 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-65cca099-aa2d-4671-8d82-8de149455596 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:18:53.951231 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-65cca099-aa2d-4671-8d82-8de149455596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:18:53.986986 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 21/40] 10.222.0.85 () {58 vars in 1361 bytes} [Tue Jun 7 14:18:53 2022] PUT /identity/v3/domains/default/users/3bd6d2364d1d4e71977a1f04d01cd108/roles/9baff2b18f2649f7b757ff2e1f63c853 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:18:53.986986 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f921754-ee17-4b2a-be76-7761506014ea None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:18:53.986986 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f921754-ee17-4b2a-be76-7761506014ea None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:18:53.986986 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f921754-ee17-4b2a-be76-7761506014ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:18:54.012637 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-7f921754-ee17-4b2a-be76-7761506014ea None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:18:54.014583 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-7f921754-ee17-4b2a-be76-7761506014ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:18:54.070043 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 20/41] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:18:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:18:54.071317 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cca099-aa2d-4671-8d82-8de149455596 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:18:54.150930 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 22/42] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:18:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:18:54.152831 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f921754-ee17-4b2a-be76-7761506014ea None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:18:54.235715 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:18:54.281648 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cca099-aa2d-4671-8d82-8de149455596 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:18:54.317490 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f921754-ee17-4b2a-be76-7761506014ea None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:18:54.321249 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:18:54.340525 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fb71ed-dc5c-4d85-b4b3-76f0dad9c031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:18:54.341229 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fb71ed-dc5c-4d85-b4b3-76f0dad9c031 None admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:18:54.341538 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fb71ed-dc5c-4d85-b4b3-76f0dad9c031 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:18:54.341819 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fb71ed-dc5c-4d85-b4b3-76f0dad9c031 None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:18:54.342618 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fb71ed-dc5c-4d85-b4b3-76f0dad9c031 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:18:54.344207 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fb71ed-dc5c-4d85-b4b3-76f0dad9c031 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:18:54.424263 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fba337-7acd-49b7-a26d-e84c2d8e46be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:18:54.425149 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fba337-7acd-49b7-a26d-e84c2d8e46be None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:18:54.425446 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fba337-7acd-49b7-a26d-e84c2d8e46be None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:18:54.425740 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fba337-7acd-49b7-a26d-e84c2d8e46be None admin] PATH_INFO: `/v3/domains/default` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:18:54.427305 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fba337-7acd-49b7-a26d-e84c2d8e46be None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:18:54.429262 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fba337-7acd-49b7-a26d-e84c2d8e46be None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:18:54.434555 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 23/43] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:18:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:18:54.435220 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fba337-7acd-49b7-a26d-e84c2d8e46be None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:18:54.478629 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 21/44] 10.222.0.85 () {60 vars in 1131 bytes} [Tue Jun 7 14:18:54 2022] POST /identity/v3/roles => generated 225 bytes in 407 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:18:54.479236 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fb71ed-dc5c-4d85-b4b3-76f0dad9c031 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:18:54.632512 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fba337-7acd-49b7-a26d-e84c2d8e46be None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:18:54.675925 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a39cb7-961b-404e-bced-c03a1417ef5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:18:54.676660 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a39cb7-961b-404e-bced-c03a1417ef5c None admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:18:54.676985 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a39cb7-961b-404e-bced-c03a1417ef5c None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:18:54.677252 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a39cb7-961b-404e-bced-c03a1417ef5c None admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:18:54.678034 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a39cb7-961b-404e-bced-c03a1417ef5c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:18:54.679642 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a39cb7-961b-404e-bced-c03a1417ef5c None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:18:54.692315 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fb71ed-dc5c-4d85-b4b3-76f0dad9c031 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:18:54.746092 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db41d037-395c-452c-b807-19d422ae0b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:18:54.746795 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db41d037-395c-452c-b807-19d422ae0b3c None admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:18:54.747093 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db41d037-395c-452c-b807-19d422ae0b3c None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:18:54.747376 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db41d037-395c-452c-b807-19d422ae0b3c None admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:18:54.748117 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db41d037-395c-452c-b807-19d422ae0b3c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:18:54.749719 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db41d037-395c-452c-b807-19d422ae0b3c None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:18:54.765658 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:18:54.867103 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 24/45] 10.222.0.85 () {60 vars in 1131 bytes} [Tue Jun 7 14:18:54 2022] POST /identity/v3/roles => generated 223 bytes in 432 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:18:54.919382 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 22/46] 10.222.0.85 () {60 vars in 1141 bytes} [Tue Jun 7 14:18:54 2022] POST /identity/v3/projects => generated 312 bytes in 435 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:18:56.618005 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09f6e10-72b1-41b8-bce8-0900064bfa06 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:18:56.619352 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09f6e10-72b1-41b8-bce8-0900064bfa06 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:18:56.619469 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09f6e10-72b1-41b8-bce8-0900064bfa06 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:18:56.620924 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 25/47] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:18:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:18:56.626522 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579f9879-97c5-4408-b947-6c07e706b92f None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:18:56.626960 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579f9879-97c5-4408-b947-6c07e706b92f None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:18:56.627138 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579f9879-97c5-4408-b947-6c07e706b92f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:18:56.644950 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-579f9879-97c5-4408-b947-6c07e706b92f None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:18:56.646156 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-579f9879-97c5-4408-b947-6c07e706b92f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:18:56.720587 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 23/48] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:18:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 14:18:56.744885 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09f6e10-72b1-41b8-bce8-0900064bfa06 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:18:56.780818 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:18:56.870850 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09f6e10-72b1-41b8-bce8-0900064bfa06 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:18:56.915221 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b76a269-e04d-4567-b978-9e6939bbe463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:18:56.916622 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b76a269-e04d-4567-b978-9e6939bbe463 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:18:56.916910 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b76a269-e04d-4567-b978-9e6939bbe463 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:18:56.917188 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b76a269-e04d-4567-b978-9e6939bbe463 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:18:56.930366 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b76a269-e04d-4567-b978-9e6939bbe463 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:18:56.938987 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-6b76a269-e04d-4567-b978-9e6939bbe463 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:18:56.938987 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:18:56.938987 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:18:56.938987 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:18:56.938987 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:18:56.938987 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:18:56.938987 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:18:56.938987 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:18:56.938987 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:18:56.938987 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:18:56.938987 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:18:56.938987 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:18:56.938987 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:18:56.938987 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:18:56.938987 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:18:56.938987 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:18:56.940013 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:18:56.940013 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:18:56.940013 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:18:56.940013 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:18:56.940013 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:18:56.940013 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:18:56.940013 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:18:56.940013 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:18:56.940013 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:18:56.940013 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:18:56.940013 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:18:56.940013 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:18:56.940013 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:18:56.940013 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:18:56.940013 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:18:56.940013 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:18:56.941768 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:18:56.941768 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:18:56.941768 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:18:56.941768 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:18:56.941768 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:18:56.941768 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:18:56.941768 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:18:56.941768 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:18:56.941768 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:18:56.941768 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:18:56.941768 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:18:56.941768 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:18:56.941768 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 26/49] 10.222.0.85 () {56 vars in 1096 bytes} [Tue Jun 7 14:18:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:18:56.948597 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579f9879-97c5-4408-b947-6c07e706b92f None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:18:57.060913 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579f9879-97c5-4408-b947-6c07e706b92f None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:18:57.083225 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:18:57.098216 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ef73aa-7ff4-47ac-8808-00828872a2e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:18:57.098887 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ef73aa-7ff4-47ac-8808-00828872a2e3 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:18:57.099172 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ef73aa-7ff4-47ac-8808-00828872a2e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:18:57.099447 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ef73aa-7ff4-47ac-8808-00828872a2e3 None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:18:57.100405 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ef73aa-7ff4-47ac-8808-00828872a2e3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:18:57.100790 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ef73aa-7ff4-47ac-8808-00828872a2e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:18:57.102424 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ef73aa-7ff4-47ac-8808-00828872a2e3 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:18:57.113000 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 24/50] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jun 7 14:18:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:18:57.124747 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b76a269-e04d-4567-b978-9e6939bbe463 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:18:57.228990 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:18:57.237885 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b76a269-e04d-4567-b978-9e6939bbe463 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:18:57.264990 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad83858-8a16-4555-8cba-569342e5a8ea None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:18:57.265342 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad83858-8a16-4555-8cba-569342e5a8ea None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:18:57.265632 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad83858-8a16-4555-8cba-569342e5a8ea None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:18:57.267038 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 25/51] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:18:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 14:18:57.274387 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61dc7c2-dd98-4af5-8986-bd81cb14d8dc None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:18:57.275393 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61dc7c2-dd98-4af5-8986-bd81cb14d8dc None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:18:57.275905 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61dc7c2-dd98-4af5-8986-bd81cb14d8dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:18:57.277785 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f37e9f-6736-4614-9950-e204d4db1138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:18:57.282509 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f37e9f-6736-4614-9950-e204d4db1138 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:18:57.282509 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f37e9f-6736-4614-9950-e204d4db1138 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:18:57.282509 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f37e9f-6736-4614-9950-e204d4db1138 None admin] PATH_INFO: `/v3/users/3bd6d2364d1d4e71977a1f04d01cd108` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:18:57.282509 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f37e9f-6736-4614-9950-e204d4db1138 None admin] RBAC: Authorizing `identity:get_user(user_id=3bd6d2364d1d4e71977a1f04d01cd108)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:18:57.282509 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f37e9f-6736-4614-9950-e204d4db1138 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:18:57.283913 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 27/52] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jun 7 14:18:57 2022] GET /identity/v3/users/3bd6d2364d1d4e71977a1f04d01cd108 => generated 247 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:18:57.289500 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f37e9f-6736-4614-9950-e204d4db1138 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:18:57.296727 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-e61dc7c2-dd98-4af5-8986-bd81cb14d8dc None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:18:57.297951 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-e61dc7c2-dd98-4af5-8986-bd81cb14d8dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:18:57.374441 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 26/53] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:18:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 14:18:57.401486 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61dc7c2-dd98-4af5-8986-bd81cb14d8dc None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:18:57.404987 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f37e9f-6736-4614-9950-e204d4db1138 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:18:57.457016 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4374b9-3308-4871-861d-32c3609da70e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:18:57.458266 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4374b9-3308-4871-861d-32c3609da70e None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:18:57.458731 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4374b9-3308-4871-861d-32c3609da70e None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:18:57.459195 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4374b9-3308-4871-861d-32c3609da70e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:18:57.461181 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4374b9-3308-4871-861d-32c3609da70e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:18:57.464060 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4374b9-3308-4871-861d-32c3609da70e None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:18:57.466773 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 28/54] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:18:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:18:57.472642 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4374b9-3308-4871-861d-32c3609da70e None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:18:57.508170 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:18:57.541678 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61dc7c2-dd98-4af5-8986-bd81cb14d8dc None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:18:57.557875 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:18:57.583145 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01001fd8-d8eb-4f3c-84f6-f6c19d58278c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:18:57.584219 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01001fd8-d8eb-4f3c-84f6-f6c19d58278c None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:18:57.584531 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01001fd8-d8eb-4f3c-84f6-f6c19d58278c None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:18:57.584823 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01001fd8-d8eb-4f3c-84f6-f6c19d58278c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:18:57.586020 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01001fd8-d8eb-4f3c-84f6-f6c19d58278c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:18:57.588124 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01001fd8-d8eb-4f3c-84f6-f6c19d58278c None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:18:57.589786 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 27/55] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:18:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:18:57.596637 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01001fd8-d8eb-4f3c-84f6-f6c19d58278c None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:18:57.605120 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4374b9-3308-4871-861d-32c3609da70e None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:18:57.663704 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e24cbd3-a7a3-4d58-bf11-8f0f59b7244e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:18:57.664330 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e24cbd3-a7a3-4d58-bf11-8f0f59b7244e None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:18:57.664593 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e24cbd3-a7a3-4d58-bf11-8f0f59b7244e None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:18:57.664881 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e24cbd3-a7a3-4d58-bf11-8f0f59b7244e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:18:57.666035 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e24cbd3-a7a3-4d58-bf11-8f0f59b7244e None admin] RBAC: Adding query filter params (role.id=9baff2b18f2649f7b757ff2e1f63c853, scope.domain.id=default, user.id=3bd6d2364d1d4e71977a1f04d01cd108) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:18:57.666486 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e24cbd3-a7a3-4d58-bf11-8f0f59b7244e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:18:57.668317 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e24cbd3-a7a3-4d58-bf11-8f0f59b7244e None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:18:57.675102 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 29/56] 10.222.0.85 () {56 vars in 1322 bytes} [Tue Jun 7 14:18:57 2022] GET /identity/v3/role_assignments?user.id=3bd6d2364d1d4e71977a1f04d01cd108&scope.domain.id=default&role.id=9baff2b18f2649f7b757ff2e1f63c853 => generated 530 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:18:57.720351 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01001fd8-d8eb-4f3c-84f6-f6c19d58278c None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:18:57.763718 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1c8294-b14e-4eca-9792-7e34d1243ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:18:57.764438 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c8294-b14e-4eca-9792-7e34d1243ebe None admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:18:57.764734 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c8294-b14e-4eca-9792-7e34d1243ebe None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:18:57.765017 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c8294-b14e-4eca-9792-7e34d1243ebe None admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:18:57.765752 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1c8294-b14e-4eca-9792-7e34d1243ebe None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:18:57.767224 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1c8294-b14e-4eca-9792-7e34d1243ebe None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:18:57.856335 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 28/57] 10.222.0.85 () {60 vars in 1140 bytes} [Tue Jun 7 14:18:57 2022] POST /identity/v3/projects => generated 298 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:19:00.167328 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630de696-6178-4b23-b3f1-f8d596ffe4b0 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:00.172762 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630de696-6178-4b23-b3f1-f8d596ffe4b0 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:00.172762 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630de696-6178-4b23-b3f1-f8d596ffe4b0 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:00.172762 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 30/58] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:19:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:19:00.176958 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b36bdea-00b6-4f5f-9bf5-8ab7d3e9e912 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:00.177449 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b36bdea-00b6-4f5f-9bf5-8ab7d3e9e912 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:00.177897 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b36bdea-00b6-4f5f-9bf5-8ab7d3e9e912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:00.200424 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-9b36bdea-00b6-4f5f-9bf5-8ab7d3e9e912 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:19:00.201628 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-9b36bdea-00b6-4f5f-9bf5-8ab7d3e9e912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:19:00.233076 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:00.295712 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 29/59] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:19:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:19:00.323576 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630de696-6178-4b23-b3f1-f8d596ffe4b0 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:00.434450 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:00.448266 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630de696-6178-4b23-b3f1-f8d596ffe4b0 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:00.484206 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aef306-adca-4bcb-b983-38b14814b88e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:00.485399 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aef306-adca-4bcb-b983-38b14814b88e None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:00.485893 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aef306-adca-4bcb-b983-38b14814b88e None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:00.486251 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aef306-adca-4bcb-b983-38b14814b88e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:00.487452 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6aef306-adca-4bcb-b983-38b14814b88e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:00.488941 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6aef306-adca-4bcb-b983-38b14814b88e None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:00.490572 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 31/60] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:19:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:19:00.497690 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b36bdea-00b6-4f5f-9bf5-8ab7d3e9e912 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:00.612791 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b36bdea-00b6-4f5f-9bf5-8ab7d3e9e912 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:00.651068 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cddd1d-6a2e-48be-b745-520f87b032f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:00.651746 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cddd1d-6a2e-48be-b745-520f87b032f6 None admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:00.652036 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cddd1d-6a2e-48be-b745-520f87b032f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:00.652320 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cddd1d-6a2e-48be-b745-520f87b032f6 None admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:00.653087 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cddd1d-6a2e-48be-b745-520f87b032f6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:00.654546 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cddd1d-6a2e-48be-b745-520f87b032f6 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:00.768604 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 30/61] 10.222.0.85 () {60 vars in 1132 bytes} [Tue Jun 7 14:19:00 2022] POST /identity/v3/users => generated 275 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:19:04.382825 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f211be-4d2c-4b31-839f-627ab4e3dd55 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:04.383860 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f211be-4d2c-4b31-839f-627ab4e3dd55 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:04.384351 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f211be-4d2c-4b31-839f-627ab4e3dd55 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:04.386283 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 32/62] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:19:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:19:04.391881 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87e036c-c349-4fa0-9032-8fdf4a1d9423 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:04.392879 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87e036c-c349-4fa0-9032-8fdf4a1d9423 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:04.393379 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87e036c-c349-4fa0-9032-8fdf4a1d9423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:04.421589 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-e87e036c-c349-4fa0-9032-8fdf4a1d9423 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:19:04.423176 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-e87e036c-c349-4fa0-9032-8fdf4a1d9423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:19:04.438840 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:04.452993 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a82160-5b0d-4499-acd0-0051b69c0f26 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:04.453278 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a82160-5b0d-4499-acd0-0051b69c0f26 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:04.453567 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a82160-5b0d-4499-acd0-0051b69c0f26 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:04.455006 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 33/63] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:19:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 14:19:04.461004 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54351e67-bdec-4b70-b0e9-a20a2e7b887d None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:04.461295 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54351e67-bdec-4b70-b0e9-a20a2e7b887d None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:04.461583 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54351e67-bdec-4b70-b0e9-a20a2e7b887d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:04.488927 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-54351e67-bdec-4b70-b0e9-a20a2e7b887d None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:19:04.490251 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-54351e67-bdec-4b70-b0e9-a20a2e7b887d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:19:04.552173 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 31/64] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:19:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:19:04.583316 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87e036c-c349-4fa0-9032-8fdf4a1d9423 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:04.654109 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 34/65] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:19:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 14:19:04.697425 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54351e67-bdec-4b70-b0e9-a20a2e7b887d None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:04.719427 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87e036c-c349-4fa0-9032-8fdf4a1d9423 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:04.761594 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d60553-7e99-43cb-b302-0166e55beccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:04.762569 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d60553-7e99-43cb-b302-0166e55beccd None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:04.762869 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d60553-7e99-43cb-b302-0166e55beccd None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:04.763177 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d60553-7e99-43cb-b302-0166e55beccd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:04.777691 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d60553-7e99-43cb-b302-0166e55beccd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:19:04.789598 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-35d60553-7e99-43cb-b302-0166e55beccd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 14:19:04.789598 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:19:04.789598 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:19:04.789598 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:19:04.789598 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:19:04.789598 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:19:04.789598 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:19:04.789598 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:19:04.789598 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:19:04.789598 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:19:04.789598 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:19:04.789598 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:19:04.789598 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:19:04.789598 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:19:04.789598 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:19:04.789598 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:19:04.791317 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:19:04.791317 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:19:04.791317 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:19:04.791317 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:19:04.791317 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:19:04.791317 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:19:04.791317 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:19:04.791317 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:19:04.791317 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:19:04.791317 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:19:04.791317 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:19:04.791317 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:19:04.791317 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:19:04.791317 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:19:04.791317 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:19:04.791317 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:19:04.792326 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:19:04.792326 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:19:04.792326 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:19:04.792326 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:19:04.792326 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:19:04.792326 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:19:04.792326 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:19:04.792326 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:19:04.792326 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:19:04.792326 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:19:04.792326 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 14:19:04.792326 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:19:04.794665 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 32/66] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jun 7 14:19:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:19:04.805138 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d60553-7e99-43cb-b302-0166e55beccd None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:04.831963 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:04.871348 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:04.884856 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54351e67-bdec-4b70-b0e9-a20a2e7b887d None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:04.924411 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ae6013-a410-40db-93a3-5f80a0bb7626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:04.925578 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ae6013-a410-40db-93a3-5f80a0bb7626 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:04.926227 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ae6013-a410-40db-93a3-5f80a0bb7626 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:04.926706 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ae6013-a410-40db-93a3-5f80a0bb7626 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:04.930143 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d60553-7e99-43cb-b302-0166e55beccd None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:04.939865 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ae6013-a410-40db-93a3-5f80a0bb7626 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:19:04.948748 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-55ae6013-a410-40db-93a3-5f80a0bb7626 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:19:04.948748 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:19:04.948748 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:19:04.948748 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:19:04.948748 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:19:04.948748 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:19:04.948748 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:19:04.948748 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:19:04.948748 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:19:04.948748 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:19:04.948748 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:19:04.948748 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:19:04.948748 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:19:04.948748 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:19:04.948748 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:19:04.948748 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:19:04.949756 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:19:04.949756 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:19:04.949756 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:19:04.949756 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:19:04.949756 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:19:04.949756 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:19:04.949756 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:19:04.949756 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:19:04.949756 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:19:04.949756 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:19:04.949756 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:19:04.949756 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:19:04.949756 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:19:04.949756 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:19:04.949756 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:19:04.949756 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:19:04.950503 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:19:04.950503 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:19:04.950503 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:19:04.950503 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:19:04.950503 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:19:04.950503 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:19:04.950503 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:19:04.950503 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:19:04.950503 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:19:04.950503 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:19:04.950503 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:19:04.950503 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:19:04.953147 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 35/67] 10.222.0.85 () {56 vars in 1096 bytes} [Tue Jun 7 14:19:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:19:04.955123 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc48e74-0899-4962-9f3b-4f895088f07f None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:04.955636 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc48e74-0899-4962-9f3b-4f895088f07f None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:04.956136 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc48e74-0899-4962-9f3b-4f895088f07f None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:04.957772 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 36/68] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:19:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:19:04.966141 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc48e74-0899-4962-9f3b-4f895088f07f None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:04.983919 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75fd227-729f-4e3c-8459-fb7ab6109ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:04.985169 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75fd227-729f-4e3c-8459-fb7ab6109ffb None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:04.985470 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75fd227-729f-4e3c-8459-fb7ab6109ffb None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:04.985761 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75fd227-729f-4e3c-8459-fb7ab6109ffb None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:04.986891 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75fd227-729f-4e3c-8459-fb7ab6109ffb None admin] RBAC: Adding query filter params (name=member) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:19:04.987215 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75fd227-729f-4e3c-8459-fb7ab6109ffb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:04.988682 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75fd227-729f-4e3c-8459-fb7ab6109ffb None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:04.998174 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 33/69] 10.222.0.85 () {56 vars in 1101 bytes} [Tue Jun 7 14:19:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:04.999791 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77b029c-6b87-4724-8326-ac257fe6ae01 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:05.000240 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77b029c-6b87-4724-8326-ac257fe6ae01 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:05.000812 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77b029c-6b87-4724-8326-ac257fe6ae01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:05.018603 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-a77b029c-6b87-4724-8326-ac257fe6ae01 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:19:05.019805 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-a77b029c-6b87-4724-8326-ac257fe6ae01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:19:05.100558 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc48e74-0899-4962-9f3b-4f895088f07f None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:05.106474 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:05.119643 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 34/70] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:19:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:19:05.119643 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d6c58d-8dd2-4496-865d-b889996fdda1 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:05.119643 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d6c58d-8dd2-4496-865d-b889996fdda1 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:05.119643 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d6c58d-8dd2-4496-865d-b889996fdda1 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:05.120512 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 35/71] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:19:05 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:19:05.121087 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d6c58d-8dd2-4496-865d-b889996fdda1 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:05.160518 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3429bb90-f612-469a-a769-1fb61da4182e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:05.161203 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3429bb90-f612-469a-a769-1fb61da4182e None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:05.161492 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3429bb90-f612-469a-a769-1fb61da4182e None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:05.161776 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3429bb90-f612-469a-a769-1fb61da4182e None admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:05.162543 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3429bb90-f612-469a-a769-1fb61da4182e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:19:05.162860 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3429bb90-f612-469a-a769-1fb61da4182e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:05.164289 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3429bb90-f612-469a-a769-1fb61da4182e None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:05.173819 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 37/72] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jun 7 14:19:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:05.180050 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4056fbdb-c4c7-4166-a970-afba1b627569 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:05.180280 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4056fbdb-c4c7-4166-a970-afba1b627569 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:05.180752 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4056fbdb-c4c7-4166-a970-afba1b627569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:05.199974 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-4056fbdb-c4c7-4166-a970-afba1b627569 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:19:05.201223 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-4056fbdb-c4c7-4166-a970-afba1b627569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:19:05.209406 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:05.242377 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d6c58d-8dd2-4496-865d-b889996fdda1 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:05.278545 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 38/73] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:19:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:19:05.279480 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4056fbdb-c4c7-4166-a970-afba1b627569 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:05.291435 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95adc809-e469-4068-93a0-10920d46e963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:05.292703 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95adc809-e469-4068-93a0-10920d46e963 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:05.293207 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95adc809-e469-4068-93a0-10920d46e963 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:05.293681 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95adc809-e469-4068-93a0-10920d46e963 None admin] PATH_INFO: `/v3/users/37e5547faf8a4aebb8725f07a28cd760` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:05.310676 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95adc809-e469-4068-93a0-10920d46e963 None admin] RBAC: Authorizing `identity:get_user(user_id=37e5547faf8a4aebb8725f07a28cd760)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:05.312427 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95adc809-e469-4068-93a0-10920d46e963 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:05.314345 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 36/74] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jun 7 14:19:05 2022] GET /identity/v3/users/37e5547faf8a4aebb8725f07a28cd760 => generated 275 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:05.315319 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95adc809-e469-4068-93a0-10920d46e963 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:05.388857 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:05.403507 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4056fbdb-c4c7-4166-a970-afba1b627569 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:05.432658 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95adc809-e469-4068-93a0-10920d46e963 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:05.443001 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86381ee-098f-4737-b829-d8f768455f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:05.443831 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86381ee-098f-4737-b829-d8f768455f98 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:05.444126 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86381ee-098f-4737-b829-d8f768455f98 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:05.444432 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86381ee-098f-4737-b829-d8f768455f98 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:05.456209 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86381ee-098f-4737-b829-d8f768455f98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:19:05.464445 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-a86381ee-098f-4737-b829-d8f768455f98 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 14:19:05.464445 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:19:05.464445 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:19:05.464445 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:19:05.464445 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:19:05.464445 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:19:05.464445 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:19:05.464445 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:19:05.464445 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:19:05.464445 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:19:05.464445 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:19:05.464445 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:19:05.464445 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:19:05.464445 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:19:05.464445 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:19:05.464445 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:19:05.465971 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:19:05.465971 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:19:05.465971 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:19:05.465971 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:19:05.465971 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:19:05.465971 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:19:05.465971 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:19:05.465971 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:19:05.465971 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:19:05.465971 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:19:05.465971 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:19:05.465971 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:19:05.465971 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:19:05.465971 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:19:05.465971 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:19:05.465971 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:19:05.466933 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:19:05.466933 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:19:05.466933 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:19:05.466933 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:19:05.466933 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:19:05.466933 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:19:05.466933 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:19:05.466933 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:19:05.466933 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:19:05.466933 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:19:05.466933 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 14:19:05.466933 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:19:05.466933 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 39/75] 10.222.0.85 () {56 vars in 1114 bytes} [Tue Jun 7 14:19:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:19:05.466933 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86381ee-098f-4737-b829-d8f768455f98 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:05.469143 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:05.477889 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d807ec6-0010-4874-b2cb-af0a05f37ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:05.478814 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d807ec6-0010-4874-b2cb-af0a05f37ab6 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:05.479135 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d807ec6-0010-4874-b2cb-af0a05f37ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:05.479336 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d807ec6-0010-4874-b2cb-af0a05f37ab6 None admin] PATH_INFO: `/v3/users/3bd6d2364d1d4e71977a1f04d01cd108` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:05.480689 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d807ec6-0010-4874-b2cb-af0a05f37ab6 None admin] RBAC: Authorizing `identity:get_user(user_id=3bd6d2364d1d4e71977a1f04d01cd108)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:05.488096 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d807ec6-0010-4874-b2cb-af0a05f37ab6 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:05.488096 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 37/76] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jun 7 14:19:05 2022] GET /identity/v3/users/3bd6d2364d1d4e71977a1f04d01cd108 => generated 247 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:05.488096 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d807ec6-0010-4874-b2cb-af0a05f37ab6 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:05.573127 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86381ee-098f-4737-b829-d8f768455f98 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:05.594563 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d807ec6-0010-4874-b2cb-af0a05f37ab6 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:05.608647 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c539e112-9d9d-4ce8-a01d-56dd6af507f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:05.609485 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c539e112-9d9d-4ce8-a01d-56dd6af507f1 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:05.609780 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c539e112-9d9d-4ce8-a01d-56dd6af507f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:05.610064 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c539e112-9d9d-4ce8-a01d-56dd6af507f1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:05.621763 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c539e112-9d9d-4ce8-a01d-56dd6af507f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:19:05.622061 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:05.629562 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-c539e112-9d9d-4ce8-a01d-56dd6af507f1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 14:19:05.629562 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:19:05.629562 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:19:05.629562 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:19:05.629562 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:19:05.629562 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:19:05.629562 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:19:05.629562 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:19:05.629562 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:19:05.629562 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:19:05.629562 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:19:05.629562 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:19:05.629562 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:19:05.629562 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:19:05.629562 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:19:05.629562 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:19:05.630480 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:19:05.630480 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:19:05.630480 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:19:05.630480 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:19:05.630480 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:19:05.630480 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:19:05.630480 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:19:05.630480 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:19:05.630480 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:19:05.630480 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:19:05.630480 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:19:05.630480 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:19:05.630480 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:19:05.630480 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:19:05.630480 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:19:05.630480 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:19:05.631314 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:19:05.631314 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:19:05.631314 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:19:05.631314 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:19:05.631314 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:19:05.631314 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:19:05.631314 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:19:05.631314 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:19:05.631314 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:19:05.631314 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:19:05.631314 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 14:19:05.631314 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:19:05.631314 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 40/77] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jun 7 14:19:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:19:05.632790 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6321f3a1-30f9-49dd-93b0-b16089c7fa1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:05.632790 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6321f3a1-30f9-49dd-93b0-b16089c7fa1c None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:05.632790 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6321f3a1-30f9-49dd-93b0-b16089c7fa1c None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:05.633012 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320afc91-72bc-4ba7-b574-821ad264171c None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:05.639925 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6321f3a1-30f9-49dd-93b0-b16089c7fa1c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:05.640049 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320afc91-72bc-4ba7-b574-821ad264171c None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:05.640049 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320afc91-72bc-4ba7-b574-821ad264171c None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:05.640049 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 41/78] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:19:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:19:05.640049 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320afc91-72bc-4ba7-b574-821ad264171c None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:05.703838 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:05.713456 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6321f3a1-30f9-49dd-93b0-b16089c7fa1c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:05.714926 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6321f3a1-30f9-49dd-93b0-b16089c7fa1c None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:05.757950 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320afc91-72bc-4ba7-b574-821ad264171c None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:05.794084 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f7b689-4cf3-440e-9159-2b45a495680c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:05.794769 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f7b689-4cf3-440e-9159-2b45a495680c None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:05.795063 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f7b689-4cf3-440e-9159-2b45a495680c None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:05.795350 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f7b689-4cf3-440e-9159-2b45a495680c None admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:05.796117 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f7b689-4cf3-440e-9159-2b45a495680c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:19:05.796469 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f7b689-4cf3-440e-9159-2b45a495680c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:05.797910 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f7b689-4cf3-440e-9159-2b45a495680c None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:05.807898 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 42/79] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jun 7 14:19:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:05.808798 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f7b689-4cf3-440e-9159-2b45a495680c None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:05.826913 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 38/80] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jun 7 14:19:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 14:19:05.827781 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6321f3a1-30f9-49dd-93b0-b16089c7fa1c None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:05.856123 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:05.917079 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f7b689-4cf3-440e-9159-2b45a495680c None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:05.938031 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6321f3a1-30f9-49dd-93b0-b16089c7fa1c None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:05.941140 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:05.951837 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c8a684-b288-4c4b-b5d7-84414f297092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:05.952569 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c8a684-b288-4c4b-b5d7-84414f297092 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:05.952859 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c8a684-b288-4c4b-b5d7-84414f297092 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:05.953150 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c8a684-b288-4c4b-b5d7-84414f297092 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:05.978546 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88bfb7c-facd-49a5-9117-0b265c16a727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:05.979223 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88bfb7c-facd-49a5-9117-0b265c16a727 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:05.979517 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88bfb7c-facd-49a5-9117-0b265c16a727 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:05.979800 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88bfb7c-facd-49a5-9117-0b265c16a727 None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:05.980590 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88bfb7c-facd-49a5-9117-0b265c16a727 None admin] RBAC: Adding query filter params (name=member) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:19:05.980927 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88bfb7c-facd-49a5-9117-0b265c16a727 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:05.982309 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88bfb7c-facd-49a5-9117-0b265c16a727 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:05.990803 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 39/81] 10.222.0.85 () {56 vars in 1101 bytes} [Tue Jun 7 14:19:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:05.992054 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69af67ff-d9c6-405e-a3ef-bb0d39ae8fb8 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:05.992354 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69af67ff-d9c6-405e-a3ef-bb0d39ae8fb8 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:05.992670 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69af67ff-d9c6-405e-a3ef-bb0d39ae8fb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:06.009867 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-69af67ff-d9c6-405e-a3ef-bb0d39ae8fb8 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:19:06.011086 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-69af67ff-d9c6-405e-a3ef-bb0d39ae8fb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:19:06.024706 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c8a684-b288-4c4b-b5d7-84414f297092 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:06.026204 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c8a684-b288-4c4b-b5d7-84414f297092 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:06.089437 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 40/82] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:19:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:19:06.090614 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69af67ff-d9c6-405e-a3ef-bb0d39ae8fb8 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:06.117931 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:06.137979 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 43/83] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jun 7 14:19:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 14:19:06.138840 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c8a684-b288-4c4b-b5d7-84414f297092 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:06.167299 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:06.218149 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69af67ff-d9c6-405e-a3ef-bb0d39ae8fb8 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:06.250933 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c8a684-b288-4c4b-b5d7-84414f297092 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:06.254158 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fc7d88-f18f-4ee1-a9a0-27eaa982362d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:06.255078 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fc7d88-f18f-4ee1-a9a0-27eaa982362d None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:06.255374 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fc7d88-f18f-4ee1-a9a0-27eaa982362d None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:06.255653 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fc7d88-f18f-4ee1-a9a0-27eaa982362d None admin] PATH_INFO: `/v3/users/37e5547faf8a4aebb8725f07a28cd760` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:06.256976 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fc7d88-f18f-4ee1-a9a0-27eaa982362d None admin] RBAC: Authorizing `identity:get_user(user_id=37e5547faf8a4aebb8725f07a28cd760)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:06.258459 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fc7d88-f18f-4ee1-a9a0-27eaa982362d None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:06.260162 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 41/84] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jun 7 14:19:06 2022] GET /identity/v3/users/37e5547faf8a4aebb8725f07a28cd760 => generated 275 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:06.260984 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fc7d88-f18f-4ee1-a9a0-27eaa982362d None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:06.293544 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8dd72d-5dec-41d9-a805-ee4b9d34ef31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:06.294423 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8dd72d-5dec-41d9-a805-ee4b9d34ef31 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:06.294771 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8dd72d-5dec-41d9-a805-ee4b9d34ef31 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:06.294994 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8dd72d-5dec-41d9-a805-ee4b9d34ef31 None admin] PATH_INFO: `/v3/projects/e04d4501005f4310819c6d8307f3df10` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:06.296205 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8dd72d-5dec-41d9-a805-ee4b9d34ef31 None admin] RBAC: Authorizing `identity:get_project(project_id=e04d4501005f4310819c6d8307f3df10)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:06.297737 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8dd72d-5dec-41d9-a805-ee4b9d34ef31 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:06.299465 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 44/85] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jun 7 14:19:06 2022] GET /identity/v3/projects/e04d4501005f4310819c6d8307f3df10 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:06.300254 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8dd72d-5dec-41d9-a805-ee4b9d34ef31 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:06.358225 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:06.374031 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fc7d88-f18f-4ee1-a9a0-27eaa982362d None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:06.402568 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:06.410100 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33ad0bf-ce82-4e9a-bd9c-0d58755398d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:06.410991 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ad0bf-ce82-4e9a-bd9c-0d58755398d0 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:06.412392 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ad0bf-ce82-4e9a-bd9c-0d58755398d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:06.412392 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ad0bf-ce82-4e9a-bd9c-0d58755398d0 None admin] PATH_INFO: `/v3/users/37e5547faf8a4aebb8725f07a28cd760` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:06.412729 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8dd72d-5dec-41d9-a805-ee4b9d34ef31 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:06.413166 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33ad0bf-ce82-4e9a-bd9c-0d58755398d0 None admin] RBAC: Authorizing `identity:get_user(user_id=37e5547faf8a4aebb8725f07a28cd760)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:06.414288 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33ad0bf-ce82-4e9a-bd9c-0d58755398d0 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:06.416559 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 42/86] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jun 7 14:19:06 2022] GET /identity/v3/users/37e5547faf8a4aebb8725f07a28cd760 => generated 275 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:06.417356 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33ad0bf-ce82-4e9a-bd9c-0d58755398d0 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:06.448697 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfdae08-2592-41e4-aa36-5b56b40ba099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:06.449507 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfdae08-2592-41e4-aa36-5b56b40ba099 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:06.449801 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfdae08-2592-41e4-aa36-5b56b40ba099 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:06.450085 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfdae08-2592-41e4-aa36-5b56b40ba099 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:06.451308 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfdae08-2592-41e4-aa36-5b56b40ba099 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:06.452819 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfdae08-2592-41e4-aa36-5b56b40ba099 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:06.454477 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 45/87] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:19:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:06.459366 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfdae08-2592-41e4-aa36-5b56b40ba099 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:06.523101 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33ad0bf-ce82-4e9a-bd9c-0d58755398d0 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:06.562774 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d5ba5a-a4e5-4cb1-bf93-ba86f82ccec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:06.563723 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d5ba5a-a4e5-4cb1-bf93-ba86f82ccec3 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:06.564027 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d5ba5a-a4e5-4cb1-bf93-ba86f82ccec3 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:06.564311 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d5ba5a-a4e5-4cb1-bf93-ba86f82ccec3 None admin] PATH_INFO: `/v3/projects/e04d4501005f4310819c6d8307f3df10` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:06.565722 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d5ba5a-a4e5-4cb1-bf93-ba86f82ccec3 None admin] RBAC: Authorizing `identity:get_project(project_id=e04d4501005f4310819c6d8307f3df10)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:06.567206 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d5ba5a-a4e5-4cb1-bf93-ba86f82ccec3 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:06.568977 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 43/88] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jun 7 14:19:06 2022] GET /identity/v3/projects/e04d4501005f4310819c6d8307f3df10 => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:06.569092 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfdae08-2592-41e4-aa36-5b56b40ba099 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:06.569764 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d5ba5a-a4e5-4cb1-bf93-ba86f82ccec3 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:06.600354 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:06.613007 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2824b6-a0de-435b-8366-a83602e9454e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:06.613658 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2824b6-a0de-435b-8366-a83602e9454e None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:06.613953 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2824b6-a0de-435b-8366-a83602e9454e None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:06.614252 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2824b6-a0de-435b-8366-a83602e9454e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:06.647001 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:06.688201 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2824b6-a0de-435b-8366-a83602e9454e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:06.689744 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2824b6-a0de-435b-8366-a83602e9454e None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:06.693253 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d5ba5a-a4e5-4cb1-bf93-ba86f82ccec3 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:06.729005 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371a4e30-f15d-4516-907f-52e927fd9644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:06.729727 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371a4e30-f15d-4516-907f-52e927fd9644 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:06.730123 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371a4e30-f15d-4516-907f-52e927fd9644 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:06.730497 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371a4e30-f15d-4516-907f-52e927fd9644 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:06.731400 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371a4e30-f15d-4516-907f-52e927fd9644 None admin] RBAC: Adding query filter params (role.id=3771341b3be641139e752aab690397a4, scope.project.id=e04d4501005f4310819c6d8307f3df10, user.id=37e5547faf8a4aebb8725f07a28cd760) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:19:06.731726 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371a4e30-f15d-4516-907f-52e927fd9644 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:06.733199 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371a4e30-f15d-4516-907f-52e927fd9644 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:06.742719 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 44/89] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jun 7 14:19:06 2022] GET /identity/v3/role_assignments?user.id=37e5547faf8a4aebb8725f07a28cd760&scope.project.id=e04d4501005f4310819c6d8307f3df10&role.id=3771341b3be641139e752aab690397a4 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:06.744563 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371a4e30-f15d-4516-907f-52e927fd9644 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:06.802538 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 46/90] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jun 7 14:19:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 14:19:06.803789 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2824b6-a0de-435b-8366-a83602e9454e None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:06.861547 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:06.880196 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371a4e30-f15d-4516-907f-52e927fd9644 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:06.881616 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:06.929718 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b0a301-1a3e-408f-a52c-2da17df0b127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:06.931371 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b0a301-1a3e-408f-a52c-2da17df0b127 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:06.932318 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b0a301-1a3e-408f-a52c-2da17df0b127 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:06.932671 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b0a301-1a3e-408f-a52c-2da17df0b127 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:06.939254 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2824b6-a0de-435b-8366-a83602e9454e None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:06.985958 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec48b22-5b25-4968-9008-c74919898ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:06.986672 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec48b22-5b25-4968-9008-c74919898ed1 None admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:06.986968 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec48b22-5b25-4968-9008-c74919898ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:06.987255 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec48b22-5b25-4968-9008-c74919898ed1 None admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:06.987997 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec48b22-5b25-4968-9008-c74919898ed1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:06.989523 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec48b22-5b25-4968-9008-c74919898ed1 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:07.020497 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b0a301-1a3e-408f-a52c-2da17df0b127 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:07.022455 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b0a301-1a3e-408f-a52c-2da17df0b127 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:07.089783 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 47/91] 10.222.0.85 () {60 vars in 1132 bytes} [Tue Jun 7 14:19:06 2022] POST /identity/v3/users => generated 289 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:19:07.089783 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec48b22-5b25-4968-9008-c74919898ed1 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:07.139748 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:07.161560 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 45/92] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jun 7 14:19:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 14:19:07.162306 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b0a301-1a3e-408f-a52c-2da17df0b127 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:07.243409 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec48b22-5b25-4968-9008-c74919898ed1 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:07.284796 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:07.290549 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dbac56-4a11-4b35-be42-bea6b87e28b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:07.291231 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dbac56-4a11-4b35-be42-bea6b87e28b5 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:07.291609 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dbac56-4a11-4b35-be42-bea6b87e28b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:07.291956 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dbac56-4a11-4b35-be42-bea6b87e28b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:07.293733 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dbac56-4a11-4b35-be42-bea6b87e28b5 None admin] RBAC: Adding query filter params (role.id=9baff2b18f2649f7b757ff2e1f63c853, scope.project.id=e04d4501005f4310819c6d8307f3df10, user.id=3bd6d2364d1d4e71977a1f04d01cd108) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:19:07.294237 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dbac56-4a11-4b35-be42-bea6b87e28b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:07.298355 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dbac56-4a11-4b35-be42-bea6b87e28b5 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:07.308097 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 48/93] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jun 7 14:19:07 2022] GET /identity/v3/role_assignments?user.id=3bd6d2364d1d4e71977a1f04d01cd108&scope.project.id=e04d4501005f4310819c6d8307f3df10&role.id=9baff2b18f2649f7b757ff2e1f63c853 => generated 259 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:07.309538 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dbac56-4a11-4b35-be42-bea6b87e28b5 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:07.345295 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b0a301-1a3e-408f-a52c-2da17df0b127 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:07.413726 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7069c4-a4e5-4ddd-8984-892b444c1cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:07.416600 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7069c4-a4e5-4ddd-8984-892b444c1cd4 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:07.417525 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7069c4-a4e5-4ddd-8984-892b444c1cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:07.418307 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7069c4-a4e5-4ddd-8984-892b444c1cd4 None admin] PATH_INFO: `/v3/projects/e04d4501005f4310819c6d8307f3df10` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:07.421280 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7069c4-a4e5-4ddd-8984-892b444c1cd4 None admin] RBAC: Authorizing `identity:get_project(project_id=e04d4501005f4310819c6d8307f3df10)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:07.424633 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7069c4-a4e5-4ddd-8984-892b444c1cd4 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:07.428682 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 46/94] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jun 7 14:19:07 2022] GET /identity/v3/projects/e04d4501005f4310819c6d8307f3df10 => generated 298 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:07.435555 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7069c4-a4e5-4ddd-8984-892b444c1cd4 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:07.466611 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dbac56-4a11-4b35-be42-bea6b87e28b5 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:07.524830 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:07.532075 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacec463-cb53-4367-977a-b0a921b59d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:07.534010 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacec463-cb53-4367-977a-b0a921b59d9c None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:07.539045 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacec463-cb53-4367-977a-b0a921b59d9c None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:07.539045 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacec463-cb53-4367-977a-b0a921b59d9c None admin] PATH_INFO: `/v3/projects/eaca1346708e4a3584eada3747bdec8b` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:07.539045 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacec463-cb53-4367-977a-b0a921b59d9c None admin] RBAC: Authorizing `identity:get_project(project_id=eaca1346708e4a3584eada3747bdec8b)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:07.539045 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacec463-cb53-4367-977a-b0a921b59d9c None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:07.542845 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 49/95] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jun 7 14:19:07 2022] GET /identity/v3/projects/eaca1346708e4a3584eada3747bdec8b => generated 312 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:07.549753 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacec463-cb53-4367-977a-b0a921b59d9c None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:07.606244 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7069c4-a4e5-4ddd-8984-892b444c1cd4 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:07.610598 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:07.657333 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1722a6c9-8b76-4f3e-8ad5-19ffa2d7d116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:07.658406 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1722a6c9-8b76-4f3e-8ad5-19ffa2d7d116 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:07.658917 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1722a6c9-8b76-4f3e-8ad5-19ffa2d7d116 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:07.659420 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1722a6c9-8b76-4f3e-8ad5-19ffa2d7d116 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:07.660592 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1722a6c9-8b76-4f3e-8ad5-19ffa2d7d116 None admin] RBAC: Adding query filter params (role.id=494cd000d6884a179c334848f52a3a21, scope.project.id=e04d4501005f4310819c6d8307f3df10, user.id=37e5547faf8a4aebb8725f07a28cd760) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:19:07.661112 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1722a6c9-8b76-4f3e-8ad5-19ffa2d7d116 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:07.662811 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1722a6c9-8b76-4f3e-8ad5-19ffa2d7d116 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:07.671582 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 47/96] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jun 7 14:19:07 2022] GET /identity/v3/role_assignments?user.id=37e5547faf8a4aebb8725f07a28cd760&scope.project.id=e04d4501005f4310819c6d8307f3df10&role.id=494cd000d6884a179c334848f52a3a21 => generated 259 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:19:07.683708 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacec463-cb53-4367-977a-b0a921b59d9c None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:07.724355 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa5c1ae-c325-4602-ba4f-8c189e7d2ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:07.725128 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa5c1ae-c325-4602-ba4f-8c189e7d2ba9 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:07.725422 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa5c1ae-c325-4602-ba4f-8c189e7d2ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:07.725708 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa5c1ae-c325-4602-ba4f-8c189e7d2ba9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:07.726584 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa5c1ae-c325-4602-ba4f-8c189e7d2ba9 None admin] RBAC: Adding query filter params (role.id=3771341b3be641139e752aab690397a4, scope.project.id=eaca1346708e4a3584eada3747bdec8b, user.id=37e5547faf8a4aebb8725f07a28cd760) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:19:07.726997 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa5c1ae-c325-4602-ba4f-8c189e7d2ba9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:07.729110 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa5c1ae-c325-4602-ba4f-8c189e7d2ba9 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:07.736284 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 50/97] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jun 7 14:19:07 2022] GET /identity/v3/role_assignments?user.id=37e5547faf8a4aebb8725f07a28cd760&scope.project.id=eaca1346708e4a3584eada3747bdec8b&role.id=3771341b3be641139e752aab690397a4 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:19:09.574185 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecd3447-fa7f-4990-a85f-452e20b9b51c None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:09.575197 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecd3447-fa7f-4990-a85f-452e20b9b51c None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:09.575785 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecd3447-fa7f-4990-a85f-452e20b9b51c None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:09.577427 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 48/98] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:19:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:19:09.589874 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5c0e0a-b2e9-4618-bb79-a31641ad27fb None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:09.590810 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5c0e0a-b2e9-4618-bb79-a31641ad27fb None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:09.591292 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5c0e0a-b2e9-4618-bb79-a31641ad27fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:09.615197 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-6b5c0e0a-b2e9-4618-bb79-a31641ad27fb None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:19:09.616881 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-6b5c0e0a-b2e9-4618-bb79-a31641ad27fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:19:09.703467 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 51/99] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:19:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 14:19:09.734243 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecd3447-fa7f-4990-a85f-452e20b9b51c None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:09.884603 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecd3447-fa7f-4990-a85f-452e20b9b51c None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:09.908607 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:09.934203 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db40d22c-2d60-4f48-9290-0b30b1fa27be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:09.935028 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db40d22c-2d60-4f48-9290-0b30b1fa27be None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:09.935320 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db40d22c-2d60-4f48-9290-0b30b1fa27be None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:09.935601 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db40d22c-2d60-4f48-9290-0b30b1fa27be None admin] PATH_INFO: `/v3/roles/member` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:09.951043 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db40d22c-2d60-4f48-9290-0b30b1fa27be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:19:09.959017 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-db40d22c-2d60-4f48-9290-0b30b1fa27be None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 14:19:09.959017 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:19:09.959017 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:19:09.959017 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:19:09.959017 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:19:09.959017 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:19:09.959017 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:19:09.959017 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:19:09.959017 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:19:09.959017 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:19:09.959017 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:19:09.959017 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:19:09.959017 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:19:09.959017 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:19:09.959017 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:19:09.959017 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:19:09.959848 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:19:09.959848 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:19:09.959848 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:19:09.959848 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:19:09.959848 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:19:09.959848 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:19:09.959848 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:19:09.959848 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:19:09.959848 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:19:09.959848 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:19:09.959848 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:19:09.959848 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:19:09.959848 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:19:09.959848 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:19:09.959848 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:19:09.959848 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:19:09.960628 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:19:09.960628 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:19:09.960628 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:19:09.960628 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:19:09.960628 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:19:09.960628 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:19:09.960628 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:19:09.960628 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:19:09.960628 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:19:09.960628 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:19:09.960628 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 14:19:09.960628 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:19:09.960628 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 49/100] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jun 7 14:19:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:19:09.968765 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5c0e0a-b2e9-4618-bb79-a31641ad27fb None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:10.041028 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:10.126101 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5c0e0a-b2e9-4618-bb79-a31641ad27fb None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:10.173671 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5981e873-fcb9-4a95-9f16-edc40d6f8cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:10.174339 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5981e873-fcb9-4a95-9f16-edc40d6f8cce None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:10.174639 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5981e873-fcb9-4a95-9f16-edc40d6f8cce None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:10.174930 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5981e873-fcb9-4a95-9f16-edc40d6f8cce None admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:10.175761 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5981e873-fcb9-4a95-9f16-edc40d6f8cce None admin] RBAC: Adding query filter params (name=member) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:19:10.176165 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5981e873-fcb9-4a95-9f16-edc40d6f8cce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:10.177729 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5981e873-fcb9-4a95-9f16-edc40d6f8cce None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:10.192567 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 52/101] 10.222.0.85 () {56 vars in 1101 bytes} [Tue Jun 7 14:19:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:10.193832 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db40d22c-2d60-4f48-9290-0b30b1fa27be None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:10.429026 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db40d22c-2d60-4f48-9290-0b30b1fa27be None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:10.482176 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857cc079-e5aa-4449-89ff-12e437c5d139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:10.483097 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857cc079-e5aa-4449-89ff-12e437c5d139 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:10.483391 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857cc079-e5aa-4449-89ff-12e437c5d139 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:10.483672 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857cc079-e5aa-4449-89ff-12e437c5d139 None admin] PATH_INFO: `/v3/users/37e5547faf8a4aebb8725f07a28cd760` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:10.484950 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857cc079-e5aa-4449-89ff-12e437c5d139 None admin] RBAC: Authorizing `identity:get_user(user_id=37e5547faf8a4aebb8725f07a28cd760)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:10.486411 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857cc079-e5aa-4449-89ff-12e437c5d139 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:10.488071 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 50/102] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jun 7 14:19:10 2022] GET /identity/v3/users/37e5547faf8a4aebb8725f07a28cd760 => generated 275 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:10.494667 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5981e873-fcb9-4a95-9f16-edc40d6f8cce None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:10.561278 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bd8d43-3437-4dce-be50-ba2523b28998 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:10.561597 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bd8d43-3437-4dce-be50-ba2523b28998 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:10.562055 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bd8d43-3437-4dce-be50-ba2523b28998 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:10.563921 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 51/103] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:19:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:19:10.579829 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dc55f7-fe6a-4251-889e-f95acf9dd33b None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:10.580599 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dc55f7-fe6a-4251-889e-f95acf9dd33b None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:10.580827 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dc55f7-fe6a-4251-889e-f95acf9dd33b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:10.614157 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-56dc55f7-fe6a-4251-889e-f95acf9dd33b None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:19:10.615437 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-56dc55f7-fe6a-4251-889e-f95acf9dd33b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:19:10.683792 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5981e873-fcb9-4a95-9f16-edc40d6f8cce None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:10.688897 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:10.721750 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:10.742526 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e44af1-5570-4943-9d9c-3a0cc226894f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:10.743465 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e44af1-5570-4943-9d9c-3a0cc226894f None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:10.743753 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e44af1-5570-4943-9d9c-3a0cc226894f None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:10.744542 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e44af1-5570-4943-9d9c-3a0cc226894f None admin] PATH_INFO: `/v3/projects/e04d4501005f4310819c6d8307f3df10` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:10.750940 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e44af1-5570-4943-9d9c-3a0cc226894f None admin] RBAC: Authorizing `identity:get_project(project_id=e04d4501005f4310819c6d8307f3df10)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:10.752667 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e44af1-5570-4943-9d9c-3a0cc226894f None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:10.754557 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 53/104] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jun 7 14:19:10 2022] GET /identity/v3/projects/e04d4501005f4310819c6d8307f3df10 => generated 298 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:10.755913 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce4d5d7-0f87-4685-8e9a-78b8c7513e7b None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:10.756206 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce4d5d7-0f87-4685-8e9a-78b8c7513e7b None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:10.756516 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce4d5d7-0f87-4685-8e9a-78b8c7513e7b None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:10.758148 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 54/105] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:19:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:19:10.767914 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f40f14b-43e9-42c1-a55b-356fd82da528 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:10.771201 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f40f14b-43e9-42c1-a55b-356fd82da528 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:10.771515 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f40f14b-43e9-42c1-a55b-356fd82da528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:10.791757 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 52/106] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:19:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:19:10.792755 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dc55f7-fe6a-4251-889e-f95acf9dd33b None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:10.807199 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-9f40f14b-43e9-42c1-a55b-356fd82da528 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:19:10.808419 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-9f40f14b-43e9-42c1-a55b-356fd82da528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:19:10.949659 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 55/107] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:19:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:19:10.950474 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f40f14b-43e9-42c1-a55b-356fd82da528 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:11.011993 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dc55f7-fe6a-4251-889e-f95acf9dd33b None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:11.086225 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f40f14b-43e9-42c1-a55b-356fd82da528 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:11.088481 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:11.097232 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065eb15f-2570-472b-ba62-dae62f202ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:11.098275 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065eb15f-2570-472b-ba62-dae62f202ffb None admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:11.098625 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065eb15f-2570-472b-ba62-dae62f202ffb None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:11.102231 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065eb15f-2570-472b-ba62-dae62f202ffb None admin] PATH_INFO: `/v3/projects/e04d4501005f4310819c6d8307f3df10/users/37e5547faf8a4aebb8725f07a28cd760/roles/3771341b3be641139e752aab690397a4` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:11.102231 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065eb15f-2570-472b-ba62-dae62f202ffb None admin] RBAC: Authorizing `identity:create_grant(project_id=e04d4501005f4310819c6d8307f3df10, user_id=37e5547faf8a4aebb8725f07a28cd760, role_id=3771341b3be641139e752aab690397a4)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:11.103525 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065eb15f-2570-472b-ba62-dae62f202ffb None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:11.127683 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f60c43-1428-4e24-82d7-7c9ce0e0c0db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:11.128828 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f60c43-1428-4e24-82d7-7c9ce0e0c0db None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:11.129257 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f60c43-1428-4e24-82d7-7c9ce0e0c0db None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:11.129675 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f60c43-1428-4e24-82d7-7c9ce0e0c0db None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:11.141992 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f60c43-1428-4e24-82d7-7c9ce0e0c0db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:19:11.150359 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-e9f60c43-1428-4e24-82d7-7c9ce0e0c0db None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:19:11.150359 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:19:11.150359 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:19:11.150359 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:19:11.150359 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:19:11.150359 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:19:11.150359 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:19:11.150359 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:19:11.150359 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:19:11.150359 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:19:11.150359 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:19:11.150359 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:19:11.150359 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:19:11.150359 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:19:11.150359 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:19:11.150359 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:19:11.151571 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:19:11.151571 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:19:11.151571 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:19:11.151571 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:19:11.151571 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:19:11.151571 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:19:11.151571 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:19:11.151571 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:19:11.151571 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:19:11.151571 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:19:11.151571 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:19:11.151571 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:19:11.151571 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:19:11.151571 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:19:11.151571 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:19:11.151571 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:19:11.152523 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:19:11.152523 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:19:11.152523 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:19:11.152523 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:19:11.152523 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:19:11.152523 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:19:11.152523 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:19:11.152523 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:19:11.152523 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:19:11.152523 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:19:11.152523 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:19:11.152523 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:19:11.154588 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 56/108] 10.222.0.85 () {56 vars in 1096 bytes} [Tue Jun 7 14:19:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:19:11.155550 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f60c43-1428-4e24-82d7-7c9ce0e0c0db None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:11.210418 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 53/109] 10.222.0.85 () {58 vars in 1439 bytes} [Tue Jun 7 14:19:10 2022] PUT /identity/v3/projects/e04d4501005f4310819c6d8307f3df10/users/37e5547faf8a4aebb8725f07a28cd760/roles/3771341b3be641139e752aab690397a4 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:19:11.211420 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065eb15f-2570-472b-ba62-dae62f202ffb None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:11.258204 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:11.272723 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f60c43-1428-4e24-82d7-7c9ce0e0c0db None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:11.312890 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356c746f-d911-490d-8cd9-722f2c95b6c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:11.313724 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356c746f-d911-490d-8cd9-722f2c95b6c5 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:11.314017 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356c746f-d911-490d-8cd9-722f2c95b6c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:11.314302 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356c746f-d911-490d-8cd9-722f2c95b6c5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:11.327002 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356c746f-d911-490d-8cd9-722f2c95b6c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:19:11.338087 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065eb15f-2570-472b-ba62-dae62f202ffb None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:11.338276 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-356c746f-d911-490d-8cd9-722f2c95b6c5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 14:19:11.338276 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:19:11.338276 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:19:11.338276 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:19:11.338276 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:19:11.338276 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:19:11.338276 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:19:11.338276 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:19:11.338276 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:19:11.338276 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:19:11.338276 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:19:11.338276 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:19:11.338276 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:19:11.338276 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:19:11.338276 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:19:11.338276 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:19:11.339064 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:19:11.339064 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:19:11.339064 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:19:11.339064 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:19:11.339064 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:19:11.339064 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:19:11.339064 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:19:11.339064 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:19:11.339064 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:19:11.339064 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:19:11.339064 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:19:11.339064 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:19:11.339064 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:19:11.339064 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:19:11.339064 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:19:11.339064 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:19:11.339791 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:19:11.339791 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:19:11.339791 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:19:11.339791 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:19:11.339791 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:19:11.339791 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:19:11.339791 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:19:11.339791 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:19:11.339791 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:19:11.339791 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:19:11.339791 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 14:19:11.339791 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:19:11.339791 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 57/110] 10.222.0.85 () {56 vars in 1114 bytes} [Tue Jun 7 14:19:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:19:11.341035 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef64fd5-956b-4e11-8359-26f1dbd44269 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:11.341335 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef64fd5-956b-4e11-8359-26f1dbd44269 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:11.341619 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef64fd5-956b-4e11-8359-26f1dbd44269 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:11.343001 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 58/111] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:19:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:19:11.345525 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547a6c2b-1fb2-4e0a-81a1-4a90803f427f None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:11.345902 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547a6c2b-1fb2-4e0a-81a1-4a90803f427f None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:11.346120 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547a6c2b-1fb2-4e0a-81a1-4a90803f427f None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:11.347552 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 59/112] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:19:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:19:11.348303 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547a6c2b-1fb2-4e0a-81a1-4a90803f427f None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:11.385976 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:11.391693 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9325429-f135-4077-a6d1-6a72352d583d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:11.392711 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9325429-f135-4077-a6d1-6a72352d583d None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:11.397086 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9325429-f135-4077-a6d1-6a72352d583d None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:11.397086 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9325429-f135-4077-a6d1-6a72352d583d None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:11.397086 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9325429-f135-4077-a6d1-6a72352d583d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:19:11.397086 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9325429-f135-4077-a6d1-6a72352d583d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:11.397086 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9325429-f135-4077-a6d1-6a72352d583d None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:11.407456 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 54/113] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jun 7 14:19:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:11.409306 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979e918c-3c3b-4a03-b399-fd5b1b9677e1 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:11.409756 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979e918c-3c3b-4a03-b399-fd5b1b9677e1 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:11.410204 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979e918c-3c3b-4a03-b399-fd5b1b9677e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:11.430537 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-979e918c-3c3b-4a03-b399-fd5b1b9677e1 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:19:11.432733 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-979e918c-3c3b-4a03-b399-fd5b1b9677e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:19:11.492838 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547a6c2b-1fb2-4e0a-81a1-4a90803f427f None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:11.534090 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef1a7d8-117c-4c39-9ca1-ad80d1659029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:11.534910 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef1a7d8-117c-4c39-9ca1-ad80d1659029 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:11.535301 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef1a7d8-117c-4c39-9ca1-ad80d1659029 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:11.535704 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef1a7d8-117c-4c39-9ca1-ad80d1659029 None admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:11.536621 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef1a7d8-117c-4c39-9ca1-ad80d1659029 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:19:11.537095 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef1a7d8-117c-4c39-9ca1-ad80d1659029 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:11.537244 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 55/114] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:19:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:19:11.538619 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef1a7d8-117c-4c39-9ca1-ad80d1659029 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:11.538937 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1548d2-d3bd-47af-a096-1b238c90f677 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:11.539361 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1548d2-d3bd-47af-a096-1b238c90f677 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:11.539785 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1548d2-d3bd-47af-a096-1b238c90f677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:11.558361 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 60/115] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jun 7 14:19:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:11.559222 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef1a7d8-117c-4c39-9ca1-ad80d1659029 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:11.575890 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-dc1548d2-d3bd-47af-a096-1b238c90f677 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:19:11.577349 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-dc1548d2-d3bd-47af-a096-1b238c90f677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:19:11.607871 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:11.654654 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:11.670614 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 56/116] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:19:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:19:11.671340 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1548d2-d3bd-47af-a096-1b238c90f677 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:11.677295 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef1a7d8-117c-4c39-9ca1-ad80d1659029 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:11.730588 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67a8145-c391-4e96-a19d-a28da76d2565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:11.732114 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67a8145-c391-4e96-a19d-a28da76d2565 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:11.732577 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67a8145-c391-4e96-a19d-a28da76d2565 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:11.732965 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67a8145-c391-4e96-a19d-a28da76d2565 None admin] PATH_INFO: `/v3/users/3bd6d2364d1d4e71977a1f04d01cd108` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:11.734617 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67a8145-c391-4e96-a19d-a28da76d2565 None admin] RBAC: Authorizing `identity:get_user(user_id=3bd6d2364d1d4e71977a1f04d01cd108)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:11.736762 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67a8145-c391-4e96-a19d-a28da76d2565 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:11.739057 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 61/117] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jun 7 14:19:11 2022] GET /identity/v3/users/3bd6d2364d1d4e71977a1f04d01cd108 => generated 247 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:11.741151 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67a8145-c391-4e96-a19d-a28da76d2565 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:11.792274 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1548d2-d3bd-47af-a096-1b238c90f677 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:11.830881 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a9846f-1ecf-4085-bc12-1a6d3662e3a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:11.832053 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a9846f-1ecf-4085-bc12-1a6d3662e3a5 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:11.832537 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a9846f-1ecf-4085-bc12-1a6d3662e3a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:11.832957 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a9846f-1ecf-4085-bc12-1a6d3662e3a5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:11.845093 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a9846f-1ecf-4085-bc12-1a6d3662e3a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:19:11.853379 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-33a9846f-1ecf-4085-bc12-1a6d3662e3a5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 14:19:11.853379 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:19:11.853379 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:19:11.853379 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:19:11.853379 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:19:11.853379 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:19:11.853379 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:19:11.853379 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:19:11.853379 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:19:11.853379 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:19:11.853379 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:19:11.853379 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:19:11.853379 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:19:11.853379 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:19:11.853379 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:19:11.853379 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:19:11.854941 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:19:11.854941 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:19:11.854941 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:19:11.854941 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:19:11.854941 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:19:11.854941 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:19:11.854941 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:19:11.854941 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:19:11.854941 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:19:11.854941 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:19:11.854941 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:19:11.854941 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:19:11.854941 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:19:11.854941 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:19:11.854941 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:19:11.854941 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:19:11.855990 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:19:11.855990 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:19:11.855990 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:19:11.855990 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:19:11.855990 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:19:11.855990 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:19:11.855990 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:19:11.855990 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:19:11.855990 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:19:11.855990 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:19:11.855990 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 14:19:11.855990 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:19:11.858990 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 57/118] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jun 7 14:19:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:19:11.860421 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a9846f-1ecf-4085-bc12-1a6d3662e3a5 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:11.885665 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:11.890110 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67a8145-c391-4e96-a19d-a28da76d2565 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:11.925696 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b415b376-01a2-43c9-a938-f2ae66fe5845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:11.926576 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b415b376-01a2-43c9-a938-f2ae66fe5845 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:11.926861 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b415b376-01a2-43c9-a938-f2ae66fe5845 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:11.927138 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b415b376-01a2-43c9-a938-f2ae66fe5845 None admin] PATH_INFO: `/v3/users/37e5547faf8a4aebb8725f07a28cd760` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:11.928343 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b415b376-01a2-43c9-a938-f2ae66fe5845 None admin] RBAC: Authorizing `identity:get_user(user_id=37e5547faf8a4aebb8725f07a28cd760)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:11.929986 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b415b376-01a2-43c9-a938-f2ae66fe5845 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:11.931776 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 62/119] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jun 7 14:19:11 2022] GET /identity/v3/users/37e5547faf8a4aebb8725f07a28cd760 => generated 275 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:11.932363 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b415b376-01a2-43c9-a938-f2ae66fe5845 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:11.964930 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:11.980800 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a9846f-1ecf-4085-bc12-1a6d3662e3a5 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:12.016530 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f18a612-051c-4c8b-bc26-acb30fd802c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:12.017672 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18a612-051c-4c8b-bc26-acb30fd802c0 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:12.017672 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18a612-051c-4c8b-bc26-acb30fd802c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:12.017918 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18a612-051c-4c8b-bc26-acb30fd802c0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:12.019104 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f18a612-051c-4c8b-bc26-acb30fd802c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:12.020604 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f18a612-051c-4c8b-bc26-acb30fd802c0 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:12.027554 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 58/120] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:19:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:12.027554 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efd89ec-a7ac-4618-a08f-e6bfdb22dd48 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:12.027554 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efd89ec-a7ac-4618-a08f-e6bfdb22dd48 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:12.027554 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efd89ec-a7ac-4618-a08f-e6bfdb22dd48 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:12.027554 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 59/121] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:19:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:19:12.027554 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efd89ec-a7ac-4618-a08f-e6bfdb22dd48 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:12.055314 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b415b376-01a2-43c9-a938-f2ae66fe5845 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:12.093611 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52df7aac-4812-4001-9632-4c5f9a692770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:12.094487 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52df7aac-4812-4001-9632-4c5f9a692770 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:12.094780 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52df7aac-4812-4001-9632-4c5f9a692770 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:12.095238 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52df7aac-4812-4001-9632-4c5f9a692770 None admin] PATH_INFO: `/v3/projects/e04d4501005f4310819c6d8307f3df10` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:12.096538 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52df7aac-4812-4001-9632-4c5f9a692770 None admin] RBAC: Authorizing `identity:get_project(project_id=e04d4501005f4310819c6d8307f3df10)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:12.098113 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52df7aac-4812-4001-9632-4c5f9a692770 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:12.104702 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 63/122] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jun 7 14:19:11 2022] GET /identity/v3/projects/e04d4501005f4310819c6d8307f3df10 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:12.104702 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52df7aac-4812-4001-9632-4c5f9a692770 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:12.131928 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:12.148338 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efd89ec-a7ac-4618-a08f-e6bfdb22dd48 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:12.194076 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc053ba0-e489-49e7-977c-33372e343604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:12.194755 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc053ba0-e489-49e7-977c-33372e343604 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:12.195046 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc053ba0-e489-49e7-977c-33372e343604 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:12.195328 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc053ba0-e489-49e7-977c-33372e343604 None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:12.196112 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc053ba0-e489-49e7-977c-33372e343604 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:19:12.197027 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc053ba0-e489-49e7-977c-33372e343604 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:12.198450 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc053ba0-e489-49e7-977c-33372e343604 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:12.207206 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 60/123] 10.222.0.85 () {56 vars in 1101 bytes} [Tue Jun 7 14:19:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:12.209399 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc053ba0-e489-49e7-977c-33372e343604 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:12.224595 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:12.240191 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52df7aac-4812-4001-9632-4c5f9a692770 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:12.281804 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf173b8-a599-46f1-985e-b6b0549edf6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:12.282732 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf173b8-a599-46f1-985e-b6b0549edf6b None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:12.283064 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf173b8-a599-46f1-985e-b6b0549edf6b None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:12.283376 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf173b8-a599-46f1-985e-b6b0549edf6b None admin] PATH_INFO: `/v3/projects/e04d4501005f4310819c6d8307f3df10` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:12.284657 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf173b8-a599-46f1-985e-b6b0549edf6b None admin] RBAC: Authorizing `identity:get_project(project_id=e04d4501005f4310819c6d8307f3df10)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:12.286136 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf173b8-a599-46f1-985e-b6b0549edf6b None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:12.292428 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 64/124] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jun 7 14:19:12 2022] GET /identity/v3/projects/e04d4501005f4310819c6d8307f3df10 => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:12.292428 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080df77a-5502-4def-9269-241feb4f3e7e None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:12.292428 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080df77a-5502-4def-9269-241feb4f3e7e None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:12.292428 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080df77a-5502-4def-9269-241feb4f3e7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:12.310122 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-080df77a-5502-4def-9269-241feb4f3e7e None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:19:12.311364 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-080df77a-5502-4def-9269-241feb4f3e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:19:12.320621 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc053ba0-e489-49e7-977c-33372e343604 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:12.356188 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee6bcfd-0c50-4418-9049-dac37918a878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:12.356883 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee6bcfd-0c50-4418-9049-dac37918a878 None admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:12.357182 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee6bcfd-0c50-4418-9049-dac37918a878 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:12.357463 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee6bcfd-0c50-4418-9049-dac37918a878 None admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:12.358192 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee6bcfd-0c50-4418-9049-dac37918a878 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:12.359636 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee6bcfd-0c50-4418-9049-dac37918a878 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:12.386607 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 65/125] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:19:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:19:12.387713 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080df77a-5502-4def-9269-241feb4f3e7e None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:12.450250 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:12.457229 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 61/126] 10.222.0.85 () {60 vars in 1141 bytes} [Tue Jun 7 14:19:12 2022] POST /identity/v3/projects => generated 302 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:19:12.458016 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee6bcfd-0c50-4418-9049-dac37918a878 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:12.509450 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:12.530381 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080df77a-5502-4def-9269-241feb4f3e7e None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:12.580983 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee6bcfd-0c50-4418-9049-dac37918a878 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:12.582241 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b34124-ecfa-4bf3-83f6-bd7286100013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:12.583610 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b34124-ecfa-4bf3-83f6-bd7286100013 None admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:12.584900 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b34124-ecfa-4bf3-83f6-bd7286100013 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:12.588727 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b34124-ecfa-4bf3-83f6-bd7286100013 None admin] PATH_INFO: `/v3/projects/e04d4501005f4310819c6d8307f3df10/users/3bd6d2364d1d4e71977a1f04d01cd108/roles/9baff2b18f2649f7b757ff2e1f63c853` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:12.592133 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b34124-ecfa-4bf3-83f6-bd7286100013 None admin] RBAC: Authorizing `identity:create_grant(project_id=e04d4501005f4310819c6d8307f3df10, user_id=3bd6d2364d1d4e71977a1f04d01cd108, role_id=9baff2b18f2649f7b757ff2e1f63c853)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:12.594784 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b34124-ecfa-4bf3-83f6-bd7286100013 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:12.624217 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9747c2d-746a-4f1e-8ecb-09e002ae7b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:12.625265 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9747c2d-746a-4f1e-8ecb-09e002ae7b64 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:12.625630 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9747c2d-746a-4f1e-8ecb-09e002ae7b64 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:12.625987 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9747c2d-746a-4f1e-8ecb-09e002ae7b64 None admin] PATH_INFO: `/v3/users/b228c514ccb9432f8d49131bfd6127be` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:12.641271 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9747c2d-746a-4f1e-8ecb-09e002ae7b64 None admin] RBAC: Authorizing `identity:get_user(user_id=b228c514ccb9432f8d49131bfd6127be)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:12.642884 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9747c2d-746a-4f1e-8ecb-09e002ae7b64 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:12.645065 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 62/127] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jun 7 14:19:12 2022] GET /identity/v3/users/b228c514ccb9432f8d49131bfd6127be => generated 289 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:12.650200 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9747c2d-746a-4f1e-8ecb-09e002ae7b64 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:12.715070 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:12.720789 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 66/128] 10.222.0.85 () {58 vars in 1439 bytes} [Tue Jun 7 14:19:12 2022] PUT /identity/v3/projects/e04d4501005f4310819c6d8307f3df10/users/3bd6d2364d1d4e71977a1f04d01cd108/roles/9baff2b18f2649f7b757ff2e1f63c853 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:19:12.721911 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b34124-ecfa-4bf3-83f6-bd7286100013 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:12.771635 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9747c2d-746a-4f1e-8ecb-09e002ae7b64 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:12.814628 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc042d3-6cc9-4bc4-a5cf-28ea0512478e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:12.815533 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc042d3-6cc9-4bc4-a5cf-28ea0512478e None admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:12.815827 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc042d3-6cc9-4bc4-a5cf-28ea0512478e None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:12.816114 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc042d3-6cc9-4bc4-a5cf-28ea0512478e None admin] PATH_INFO: `/v3/projects/e04d4501005f4310819c6d8307f3df10/users/37e5547faf8a4aebb8725f07a28cd760/roles/494cd000d6884a179c334848f52a3a21` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:12.818545 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc042d3-6cc9-4bc4-a5cf-28ea0512478e None admin] RBAC: Authorizing `identity:create_grant(project_id=e04d4501005f4310819c6d8307f3df10, user_id=37e5547faf8a4aebb8725f07a28cd760, role_id=494cd000d6884a179c334848f52a3a21)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:12.820117 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc042d3-6cc9-4bc4-a5cf-28ea0512478e None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:12.866762 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b34124-ecfa-4bf3-83f6-bd7286100013 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:12.908669 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7411ec-b4e8-44a9-b143-570ee00da6b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:12.909432 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7411ec-b4e8-44a9-b143-570ee00da6b1 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:12.909716 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7411ec-b4e8-44a9-b143-570ee00da6b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:12.910053 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7411ec-b4e8-44a9-b143-570ee00da6b1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:12.911424 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 63/129] 10.222.0.85 () {58 vars in 1439 bytes} [Tue Jun 7 14:19:12 2022] PUT /identity/v3/projects/e04d4501005f4310819c6d8307f3df10/users/37e5547faf8a4aebb8725f07a28cd760/roles/494cd000d6884a179c334848f52a3a21 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:19:12.912175 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc042d3-6cc9-4bc4-a5cf-28ea0512478e None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:12.925170 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7411ec-b4e8-44a9-b143-570ee00da6b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:19:12.934377 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-0e7411ec-b4e8-44a9-b143-570ee00da6b1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 14:19:12.934377 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:19:12.934377 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:19:12.934377 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:19:12.934377 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:19:12.934377 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:19:12.934377 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:19:12.934377 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:19:12.934377 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:19:12.934377 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:19:12.934377 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:19:12.934377 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:19:12.934377 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:19:12.934377 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:19:12.934377 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:19:12.934377 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:19:12.935367 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:19:12.935367 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:19:12.935367 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:19:12.935367 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:19:12.935367 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:19:12.935367 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:19:12.935367 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:19:12.935367 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:19:12.935367 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:19:12.935367 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:19:12.935367 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:19:12.935367 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:19:12.935367 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:19:12.935367 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:19:12.935367 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:19:12.935367 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:19:12.936128 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:19:12.936128 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:19:12.936128 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:19:12.936128 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:19:12.936128 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:19:12.936128 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:19:12.936128 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:19:12.936128 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:19:12.936128 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:19:12.936128 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:19:12.936128 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 14:19:12.936128 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:19:12.936743 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 67/130] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jun 7 14:19:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:19:12.945687 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7411ec-b4e8-44a9-b143-570ee00da6b1 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:12.960224 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:13.091651 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7411ec-b4e8-44a9-b143-570ee00da6b1 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:13.124776 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc042d3-6cc9-4bc4-a5cf-28ea0512478e None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:13.134572 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c92e3a-3e1e-4a7b-8931-5fe51e4b363d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:13.135265 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c92e3a-3e1e-4a7b-8931-5fe51e4b363d None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:13.135560 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c92e3a-3e1e-4a7b-8931-5fe51e4b363d None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:13.135849 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c92e3a-3e1e-4a7b-8931-5fe51e4b363d None admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:13.138290 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c92e3a-3e1e-4a7b-8931-5fe51e4b363d None admin] RBAC: Adding query filter params (name=member) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:19:13.138621 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c92e3a-3e1e-4a7b-8931-5fe51e4b363d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:13.140067 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c92e3a-3e1e-4a7b-8931-5fe51e4b363d None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:13.150208 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 68/131] 10.222.0.85 () {56 vars in 1101 bytes} [Tue Jun 7 14:19:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:13.152149 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:13.156270 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c92e3a-3e1e-4a7b-8931-5fe51e4b363d None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:13.176297 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49f0ee7-e407-441d-8f3f-913087820e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:13.178764 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49f0ee7-e407-441d-8f3f-913087820e70 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:13.179060 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49f0ee7-e407-441d-8f3f-913087820e70 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:13.179348 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49f0ee7-e407-441d-8f3f-913087820e70 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:13.308743 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:13.342561 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c92e3a-3e1e-4a7b-8931-5fe51e4b363d None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:13.353543 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49f0ee7-e407-441d-8f3f-913087820e70 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:13.355257 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49f0ee7-e407-441d-8f3f-913087820e70 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:13.453766 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994030e8-ed0a-4bd6-a52e-1c41cd4e81bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:13.454687 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994030e8-ed0a-4bd6-a52e-1c41cd4e81bf None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:13.454979 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994030e8-ed0a-4bd6-a52e-1c41cd4e81bf None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:13.455264 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994030e8-ed0a-4bd6-a52e-1c41cd4e81bf None admin] PATH_INFO: `/v3/users/37e5547faf8a4aebb8725f07a28cd760` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:13.456534 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994030e8-ed0a-4bd6-a52e-1c41cd4e81bf None admin] RBAC: Authorizing `identity:get_user(user_id=37e5547faf8a4aebb8725f07a28cd760)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:13.458016 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994030e8-ed0a-4bd6-a52e-1c41cd4e81bf None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:13.459694 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 69/132] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jun 7 14:19:13 2022] GET /identity/v3/users/37e5547faf8a4aebb8725f07a28cd760 => generated 275 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:13.465474 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994030e8-ed0a-4bd6-a52e-1c41cd4e81bf None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:13.476986 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 64/133] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jun 7 14:19:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 565 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 14:19:13.485661 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49f0ee7-e407-441d-8f3f-913087820e70 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:13.505286 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:13.582478 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994030e8-ed0a-4bd6-a52e-1c41cd4e81bf None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:13.592727 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49f0ee7-e407-441d-8f3f-913087820e70 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:13.621537 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea560684-178f-44b6-bdc0-7a51ebda8abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:13.622713 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea560684-178f-44b6-bdc0-7a51ebda8abf None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:13.623099 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea560684-178f-44b6-bdc0-7a51ebda8abf None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:13.623546 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea560684-178f-44b6-bdc0-7a51ebda8abf None admin] PATH_INFO: `/v3/projects/eaca1346708e4a3584eada3747bdec8b` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:13.625476 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea560684-178f-44b6-bdc0-7a51ebda8abf None admin] RBAC: Authorizing `identity:get_project(project_id=eaca1346708e4a3584eada3747bdec8b)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:13.627139 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea560684-178f-44b6-bdc0-7a51ebda8abf None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:13.629819 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 70/134] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jun 7 14:19:13 2022] GET /identity/v3/projects/eaca1346708e4a3584eada3747bdec8b => generated 312 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:19:13.633588 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daed037-58a0-4785-96a8-eeaa4e26e865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:13.635111 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daed037-58a0-4785-96a8-eeaa4e26e865 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:13.635598 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daed037-58a0-4785-96a8-eeaa4e26e865 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:13.635699 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea560684-178f-44b6-bdc0-7a51ebda8abf None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:13.636239 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daed037-58a0-4785-96a8-eeaa4e26e865 None admin] PATH_INFO: `/v3/projects/e04d4501005f4310819c6d8307f3df10` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:13.637686 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daed037-58a0-4785-96a8-eeaa4e26e865 None admin] RBAC: Authorizing `identity:get_project(project_id=e04d4501005f4310819c6d8307f3df10)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:13.639334 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daed037-58a0-4785-96a8-eeaa4e26e865 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:13.639794 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:13.641485 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 65/135] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jun 7 14:19:13 2022] GET /identity/v3/projects/e04d4501005f4310819c6d8307f3df10 => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:19:13.649984 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daed037-58a0-4785-96a8-eeaa4e26e865 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:13.759921 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea560684-178f-44b6-bdc0-7a51ebda8abf None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:13.805156 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce123526-00e3-46c6-9975-fa002d67ccc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:13.806604 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce123526-00e3-46c6-9975-fa002d67ccc0 None admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:13.807853 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce123526-00e3-46c6-9975-fa002d67ccc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:13.808943 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce123526-00e3-46c6-9975-fa002d67ccc0 None admin] PATH_INFO: `/v3/projects/eaca1346708e4a3584eada3747bdec8b/users/37e5547faf8a4aebb8725f07a28cd760/roles/3771341b3be641139e752aab690397a4` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:13.818616 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce123526-00e3-46c6-9975-fa002d67ccc0 None admin] RBAC: Authorizing `identity:create_grant(project_id=eaca1346708e4a3584eada3747bdec8b, user_id=37e5547faf8a4aebb8725f07a28cd760, role_id=3771341b3be641139e752aab690397a4)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:13.820714 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce123526-00e3-46c6-9975-fa002d67ccc0 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:13.856389 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:13.885870 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daed037-58a0-4785-96a8-eeaa4e26e865 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:13.909152 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 71/136] 10.222.0.85 () {58 vars in 1439 bytes} [Tue Jun 7 14:19:13 2022] PUT /identity/v3/projects/eaca1346708e4a3584eada3747bdec8b/users/37e5547faf8a4aebb8725f07a28cd760/roles/3771341b3be641139e752aab690397a4 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:19:13.911133 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1ce692-98a5-4411-8cb5-4112e1ff900a None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:13.911591 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1ce692-98a5-4411-8cb5-4112e1ff900a None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:13.912085 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1ce692-98a5-4411-8cb5-4112e1ff900a None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:13.914226 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 72/137] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:19:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:19:13.925130 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630826d1-fd79-4590-8cee-07daaeb841b2 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:13.925765 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630826d1-fd79-4590-8cee-07daaeb841b2 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:13.926290 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630826d1-fd79-4590-8cee-07daaeb841b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:13.972533 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-630826d1-fd79-4590-8cee-07daaeb841b2 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:19:13.973778 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-630826d1-fd79-4590-8cee-07daaeb841b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:19:13.996135 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aade2568-b5f5-4cd8-9dba-cc4f06b61e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:14.001236 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade2568-b5f5-4cd8-9dba-cc4f06b61e2c None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:14.001535 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade2568-b5f5-4cd8-9dba-cc4f06b61e2c None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:14.001823 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade2568-b5f5-4cd8-9dba-cc4f06b61e2c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:14.002717 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aade2568-b5f5-4cd8-9dba-cc4f06b61e2c None admin] RBAC: Adding query filter params (role.id=e2072a2a35d64eec9d32dac7c47c4419, scope.project.id=e04d4501005f4310819c6d8307f3df10, user.id=b228c514ccb9432f8d49131bfd6127be) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:19:14.003039 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aade2568-b5f5-4cd8-9dba-cc4f06b61e2c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:14.004675 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aade2568-b5f5-4cd8-9dba-cc4f06b61e2c None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:14.019619 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 66/138] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jun 7 14:19:13 2022] GET /identity/v3/role_assignments?user.id=b228c514ccb9432f8d49131bfd6127be&scope.project.id=e04d4501005f4310819c6d8307f3df10&role.id=e2072a2a35d64eec9d32dac7c47c4419 => generated 259 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:14.096859 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:14.101687 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 73/139] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:19:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:19:14.147820 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aade2568-b5f5-4cd8-9dba-cc4f06b61e2c None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:14.283019 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aade2568-b5f5-4cd8-9dba-cc4f06b61e2c None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:14.324798 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c14e68-9867-454f-88ce-7c495e7b6565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:14.325640 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c14e68-9867-454f-88ce-7c495e7b6565 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:14.325938 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c14e68-9867-454f-88ce-7c495e7b6565 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:14.326226 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c14e68-9867-454f-88ce-7c495e7b6565 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:14.342910 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c14e68-9867-454f-88ce-7c495e7b6565 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:19:14.351409 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-a5c14e68-9867-454f-88ce-7c495e7b6565 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 14:19:14.351409 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:19:14.351409 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:19:14.351409 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:19:14.351409 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:19:14.351409 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:19:14.351409 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:19:14.351409 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:19:14.351409 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:19:14.351409 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:19:14.351409 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:19:14.351409 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:19:14.351409 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:19:14.351409 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:19:14.351409 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:19:14.351409 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:19:14.352403 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:19:14.352403 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:19:14.352403 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:19:14.352403 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:19:14.352403 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:19:14.352403 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:19:14.352403 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:19:14.352403 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:19:14.352403 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:19:14.352403 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:19:14.352403 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:19:14.352403 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:19:14.352403 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:19:14.352403 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:19:14.352403 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:19:14.352403 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:19:14.353161 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:19:14.353161 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:19:14.353161 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:19:14.353161 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:19:14.353161 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:19:14.353161 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:19:14.353161 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:19:14.353161 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:19:14.353161 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:19:14.353161 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:19:14.353161 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 14:19:14.353161 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:19:14.356937 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 67/140] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jun 7 14:19:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:19:14.363102 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630826d1-fd79-4590-8cee-07daaeb841b2 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:14.478118 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630826d1-fd79-4590-8cee-07daaeb841b2 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:14.520175 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aa180c-d8d1-4069-8d35-7f762ee178d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:14.521082 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aa180c-d8d1-4069-8d35-7f762ee178d9 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:14.521376 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aa180c-d8d1-4069-8d35-7f762ee178d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:14.521656 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aa180c-d8d1-4069-8d35-7f762ee178d9 None admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:14.522432 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aa180c-d8d1-4069-8d35-7f762ee178d9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:19:14.522747 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aa180c-d8d1-4069-8d35-7f762ee178d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:14.524152 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aa180c-d8d1-4069-8d35-7f762ee178d9 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:14.543986 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 74/141] 10.222.0.85 () {56 vars in 1101 bytes} [Tue Jun 7 14:19:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:14.549934 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c14e68-9867-454f-88ce-7c495e7b6565 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:14.569327 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:14.681439 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c14e68-9867-454f-88ce-7c495e7b6565 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:14.725121 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efda2852-9ae0-4e8c-a09b-ec2c192c0167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:14.726030 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda2852-9ae0-4e8c-a09b-ec2c192c0167 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:14.726328 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda2852-9ae0-4e8c-a09b-ec2c192c0167 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:14.726722 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda2852-9ae0-4e8c-a09b-ec2c192c0167 None admin] PATH_INFO: `/v3/users/37e5547faf8a4aebb8725f07a28cd760` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:14.728578 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efda2852-9ae0-4e8c-a09b-ec2c192c0167 None admin] RBAC: Authorizing `identity:get_user(user_id=37e5547faf8a4aebb8725f07a28cd760)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:14.730052 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efda2852-9ae0-4e8c-a09b-ec2c192c0167 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:14.732022 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 68/142] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jun 7 14:19:14 2022] GET /identity/v3/users/37e5547faf8a4aebb8725f07a28cd760 => generated 275 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:14.741534 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aa180c-d8d1-4069-8d35-7f762ee178d9 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:14.818834 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:14.865778 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aa180c-d8d1-4069-8d35-7f762ee178d9 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:14.906895 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57417577-d959-4124-bf80-2b8c818dc31f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:14.907568 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57417577-d959-4124-bf80-2b8c818dc31f None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:14.907860 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57417577-d959-4124-bf80-2b8c818dc31f None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:14.908142 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57417577-d959-4124-bf80-2b8c818dc31f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:15.001531 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57417577-d959-4124-bf80-2b8c818dc31f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:15.003029 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57417577-d959-4124-bf80-2b8c818dc31f None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:15.100968 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:15.142387 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 75/143] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jun 7 14:19:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 14:19:15.148136 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efda2852-9ae0-4e8c-a09b-ec2c192c0167 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:15.256998 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:15.282222 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efda2852-9ae0-4e8c-a09b-ec2c192c0167 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:15.332251 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cc30d6-de8f-47d4-9e37-8f8aeb64c601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:15.333332 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cc30d6-de8f-47d4-9e37-8f8aeb64c601 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:15.333629 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cc30d6-de8f-47d4-9e37-8f8aeb64c601 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:15.333971 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cc30d6-de8f-47d4-9e37-8f8aeb64c601 None admin] PATH_INFO: `/v3/projects/e04d4501005f4310819c6d8307f3df10` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:15.335341 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cc30d6-de8f-47d4-9e37-8f8aeb64c601 None admin] RBAC: Authorizing `identity:get_project(project_id=e04d4501005f4310819c6d8307f3df10)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:15.337415 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cc30d6-de8f-47d4-9e37-8f8aeb64c601 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:15.339274 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 69/144] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jun 7 14:19:15 2022] GET /identity/v3/projects/e04d4501005f4310819c6d8307f3df10 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:15.345403 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57417577-d959-4124-bf80-2b8c818dc31f None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:15.472585 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57417577-d959-4124-bf80-2b8c818dc31f None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:15.510234 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd9e1ee-6423-4a61-8b89-1e1e17c378f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:15.510911 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd9e1ee-6423-4a61-8b89-1e1e17c378f9 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:15.511264 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd9e1ee-6423-4a61-8b89-1e1e17c378f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:15.511556 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd9e1ee-6423-4a61-8b89-1e1e17c378f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:15.512440 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd9e1ee-6423-4a61-8b89-1e1e17c378f9 None admin] RBAC: Adding query filter params (role.id=3771341b3be641139e752aab690397a4, scope.project.id=e04d4501005f4310819c6d8307f3df10, user.id=37e5547faf8a4aebb8725f07a28cd760) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:19:15.512768 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd9e1ee-6423-4a61-8b89-1e1e17c378f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:15.514241 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd9e1ee-6423-4a61-8b89-1e1e17c378f9 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:15.521292 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 76/145] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jun 7 14:19:15 2022] GET /identity/v3/role_assignments?user.id=37e5547faf8a4aebb8725f07a28cd760&scope.project.id=e04d4501005f4310819c6d8307f3df10&role.id=3771341b3be641139e752aab690397a4 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:15.921973 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b32873-0fc0-40fd-a564-47209a6499f2 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:15.922621 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b32873-0fc0-40fd-a564-47209a6499f2 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:15.922621 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b32873-0fc0-40fd-a564-47209a6499f2 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:15.923844 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 70/146] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:19:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:19:15.929353 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0ab136-6e4d-40c0-b25e-e13cb1896680 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:15.929653 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0ab136-6e4d-40c0-b25e-e13cb1896680 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:15.929944 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0ab136-6e4d-40c0-b25e-e13cb1896680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:15.953708 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-1c0ab136-6e4d-40c0-b25e-e13cb1896680 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:19:15.957002 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-1c0ab136-6e4d-40c0-b25e-e13cb1896680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:19:15.976059 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:16.048106 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 77/147] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:19:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:19:16.081757 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b32873-0fc0-40fd-a564-47209a6499f2 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:16.216656 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b32873-0fc0-40fd-a564-47209a6499f2 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:16.230883 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e097f4f2-533d-43a3-92dc-12cae7562aaf None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:16.231270 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e097f4f2-533d-43a3-92dc-12cae7562aaf None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:16.231528 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e097f4f2-533d-43a3-92dc-12cae7562aaf None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:16.233138 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 78/148] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:19:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 14:19:16.247447 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca70669e-7fb9-46e0-8334-94b9581fa312 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:16.247839 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca70669e-7fb9-46e0-8334-94b9581fa312 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:16.248261 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca70669e-7fb9-46e0-8334-94b9581fa312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:16.279227 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-ca70669e-7fb9-46e0-8334-94b9581fa312 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:19:16.280453 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-ca70669e-7fb9-46e0-8334-94b9581fa312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:19:16.303401 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fabab7-b0eb-49b6-b434-62db7ab2427f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:16.304268 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fabab7-b0eb-49b6-b434-62db7ab2427f None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:16.304586 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fabab7-b0eb-49b6-b434-62db7ab2427f None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:16.304928 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fabab7-b0eb-49b6-b434-62db7ab2427f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:16.328694 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fabab7-b0eb-49b6-b434-62db7ab2427f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:19:16.341991 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-77fabab7-b0eb-49b6-b434-62db7ab2427f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:19:16.341991 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:19:16.341991 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:19:16.341991 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:19:16.341991 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:19:16.341991 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:19:16.341991 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:19:16.341991 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:19:16.341991 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:19:16.341991 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:19:16.341991 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:19:16.341991 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:19:16.341991 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:19:16.341991 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:19:16.341991 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:19:16.341991 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:19:16.342871 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:19:16.342871 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:19:16.342871 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:19:16.342871 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:19:16.342871 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:19:16.342871 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:19:16.342871 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:19:16.342871 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:19:16.342871 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:19:16.342871 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:19:16.342871 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:19:16.342871 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:19:16.342871 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:19:16.342871 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:19:16.342871 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:19:16.342871 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:19:16.343615 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:19:16.343615 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:19:16.343615 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:19:16.343615 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:19:16.343615 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:19:16.343615 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:19:16.343615 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:19:16.343615 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:19:16.343615 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:19:16.343615 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:19:16.343615 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:19:16.343615 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:19:16.349533 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 71/149] 10.222.0.85 () {56 vars in 1096 bytes} [Tue Jun 7 14:19:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:19:16.355774 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fabab7-b0eb-49b6-b434-62db7ab2427f None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:16.372738 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:16.437648 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 79/150] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:19:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:19:16.466887 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca70669e-7fb9-46e0-8334-94b9581fa312 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:16.560770 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:16.578000 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fabab7-b0eb-49b6-b434-62db7ab2427f None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:16.587403 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca70669e-7fb9-46e0-8334-94b9581fa312 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:16.644689 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3df1495-b229-470b-9241-ebda2f5b1c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:16.645367 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3df1495-b229-470b-9241-ebda2f5b1c0a None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:16.645654 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3df1495-b229-470b-9241-ebda2f5b1c0a None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:16.645933 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3df1495-b229-470b-9241-ebda2f5b1c0a None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:16.646701 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3df1495-b229-470b-9241-ebda2f5b1c0a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:19:16.647010 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3df1495-b229-470b-9241-ebda2f5b1c0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:16.648424 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3df1495-b229-470b-9241-ebda2f5b1c0a None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:16.664159 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74a3762-5277-4313-a02a-3d3d27b41c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:16.665026 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74a3762-5277-4313-a02a-3d3d27b41c78 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:16.665315 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74a3762-5277-4313-a02a-3d3d27b41c78 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:16.665591 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74a3762-5277-4313-a02a-3d3d27b41c78 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:16.668003 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 72/151] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jun 7 14:19:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:16.668862 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74a3762-5277-4313-a02a-3d3d27b41c78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:16.670444 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74a3762-5277-4313-a02a-3d3d27b41c78 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:16.672226 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 80/152] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:19:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:16.674456 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3df1495-b229-470b-9241-ebda2f5b1c0a None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:16.678217 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74a3762-5277-4313-a02a-3d3d27b41c78 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:16.819114 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:16.854520 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74a3762-5277-4313-a02a-3d3d27b41c78 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:16.892820 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b80128-04a1-4ba9-8ed2-7343bd04f2dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:16.893511 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b80128-04a1-4ba9-8ed2-7343bd04f2dd None admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:16.893804 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b80128-04a1-4ba9-8ed2-7343bd04f2dd None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:16.894087 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b80128-04a1-4ba9-8ed2-7343bd04f2dd None admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:16.894814 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b80128-04a1-4ba9-8ed2-7343bd04f2dd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:16.896291 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b80128-04a1-4ba9-8ed2-7343bd04f2dd None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:16.904766 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3df1495-b229-470b-9241-ebda2f5b1c0a None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:16.951182 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dad6289-d819-4968-9f76-7c7157a0c8f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:16.952356 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dad6289-d819-4968-9f76-7c7157a0c8f6 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:16.952665 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dad6289-d819-4968-9f76-7c7157a0c8f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:16.952947 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dad6289-d819-4968-9f76-7c7157a0c8f6 None admin] PATH_INFO: `/v3/users/3bd6d2364d1d4e71977a1f04d01cd108` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:16.956047 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dad6289-d819-4968-9f76-7c7157a0c8f6 None admin] RBAC: Authorizing `identity:get_user(user_id=3bd6d2364d1d4e71977a1f04d01cd108)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:16.958328 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dad6289-d819-4968-9f76-7c7157a0c8f6 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:16.960222 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 73/153] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jun 7 14:19:16 2022] GET /identity/v3/users/3bd6d2364d1d4e71977a1f04d01cd108 => generated 247 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:19:16.961510 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818ef797-306e-4e1b-a196-99f1e876fba8 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:16.961839 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818ef797-306e-4e1b-a196-99f1e876fba8 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:16.962114 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818ef797-306e-4e1b-a196-99f1e876fba8 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:16.963536 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 74/154] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:19:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:19:16.971963 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818ef797-306e-4e1b-a196-99f1e876fba8 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:17.022419 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 81/155] 10.222.0.85 () {60 vars in 1132 bytes} [Tue Jun 7 14:19:16 2022] POST /identity/v3/users => generated 283 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:19:17.022419 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f898da6-39a0-44d2-8e9c-32924060709e None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:17.022701 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f898da6-39a0-44d2-8e9c-32924060709e None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:17.023132 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f898da6-39a0-44d2-8e9c-32924060709e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:17.039600 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:17.064974 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-4f898da6-39a0-44d2-8e9c-32924060709e None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:19:17.067828 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-4f898da6-39a0-44d2-8e9c-32924060709e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:19:17.173018 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818ef797-306e-4e1b-a196-99f1e876fba8 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:17.210410 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 82/156] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:19:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:19:17.218025 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44713ea-7a93-4ce4-a37a-3df8ebd2a492 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:17.218324 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44713ea-7a93-4ce4-a37a-3df8ebd2a492 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:17.218592 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44713ea-7a93-4ce4-a37a-3df8ebd2a492 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:17.221475 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 83/157] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:19:17 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:19:17.230264 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27401c03-592a-41a2-be81-7fcc2302805b None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:17.230576 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27401c03-592a-41a2-be81-7fcc2302805b None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:17.230829 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27401c03-592a-41a2-be81-7fcc2302805b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:17.245831 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc064fe-ddd3-49cd-a80e-f2a29dbc0e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:17.246613 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc064fe-ddd3-49cd-a80e-f2a29dbc0e7d None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:17.246908 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc064fe-ddd3-49cd-a80e-f2a29dbc0e7d None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:17.247250 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc064fe-ddd3-49cd-a80e-f2a29dbc0e7d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:17.251598 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:17.255879 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-27401c03-592a-41a2-be81-7fcc2302805b None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:19:17.257309 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-27401c03-592a-41a2-be81-7fcc2302805b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:19:17.350180 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc064fe-ddd3-49cd-a80e-f2a29dbc0e7d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:17.351768 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc064fe-ddd3-49cd-a80e-f2a29dbc0e7d None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:17.357288 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 84/158] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:19:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:19:17.359164 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27401c03-592a-41a2-be81-7fcc2302805b None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:17.445162 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:17.497372 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 75/159] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jun 7 14:19:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 529 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 14:19:17.499028 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27401c03-592a-41a2-be81-7fcc2302805b None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:17.499272 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc064fe-ddd3-49cd-a80e-f2a29dbc0e7d None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:17.549452 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:17.550416 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b3998b-77b6-40f9-a1b5-e24cd7de989c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:17.551256 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b3998b-77b6-40f9-a1b5-e24cd7de989c None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:17.551547 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b3998b-77b6-40f9-a1b5-e24cd7de989c None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:17.551858 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b3998b-77b6-40f9-a1b5-e24cd7de989c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:17.569987 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b3998b-77b6-40f9-a1b5-e24cd7de989c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:19:17.578288 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-e0b3998b-77b6-40f9-a1b5-e24cd7de989c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 14:19:17.578288 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:19:17.578288 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:19:17.578288 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:19:17.578288 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:19:17.578288 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:19:17.578288 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:19:17.578288 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:19:17.578288 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:19:17.578288 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:19:17.578288 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:19:17.578288 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:19:17.578288 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:19:17.578288 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:19:17.578288 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:19:17.578288 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:19:17.579417 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:19:17.579417 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:19:17.579417 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:19:17.579417 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:19:17.579417 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:19:17.579417 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:19:17.579417 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:19:17.579417 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:19:17.579417 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:19:17.579417 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:19:17.579417 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:19:17.579417 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:19:17.579417 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:19:17.579417 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:19:17.579417 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:19:17.579417 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:19:17.580354 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:19:17.580354 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:19:17.580354 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:19:17.580354 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:19:17.580354 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:19:17.580354 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:19:17.580354 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:19:17.580354 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:19:17.580354 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:19:17.580354 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:19:17.580354 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 14:19:17.580354 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:19:17.580354 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 85/160] 10.222.0.85 () {56 vars in 1114 bytes} [Tue Jun 7 14:19:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:19:17.581065 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4674fb8-e0ff-4d3c-80f8-76fcc53b052e None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:17.581144 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4674fb8-e0ff-4d3c-80f8-76fcc53b052e None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:17.581502 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4674fb8-e0ff-4d3c-80f8-76fcc53b052e None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:17.583180 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 86/161] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:19:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:19:17.583357 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4674fb8-e0ff-4d3c-80f8-76fcc53b052e None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:17.661697 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc064fe-ddd3-49cd-a80e-f2a29dbc0e7d None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:17.726622 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfbccab-c2d3-469f-b8f4-e7e3e0ce208b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:17.728746 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfbccab-c2d3-469f-b8f4-e7e3e0ce208b None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:17.729310 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfbccab-c2d3-469f-b8f4-e7e3e0ce208b None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:17.729855 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfbccab-c2d3-469f-b8f4-e7e3e0ce208b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:17.736150 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4674fb8-e0ff-4d3c-80f8-76fcc53b052e None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:17.743152 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfbccab-c2d3-469f-b8f4-e7e3e0ce208b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:19:17.752261 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-ddfbccab-c2d3-469f-b8f4-e7e3e0ce208b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 14:19:17.752261 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:19:17.752261 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:19:17.752261 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:19:17.752261 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:19:17.752261 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:19:17.752261 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:19:17.752261 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:19:17.752261 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:19:17.752261 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:19:17.752261 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:19:17.752261 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:19:17.752261 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:19:17.752261 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:19:17.752261 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:19:17.752261 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:19:17.753314 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:19:17.753314 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:19:17.753314 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:19:17.753314 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:19:17.753314 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:19:17.753314 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:19:17.753314 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:19:17.753314 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:19:17.753314 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:19:17.753314 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:19:17.753314 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:19:17.753314 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:19:17.753314 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:19:17.753314 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:19:17.753314 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:19:17.753314 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:19:17.754070 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:19:17.754070 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:19:17.754070 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:19:17.754070 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:19:17.754070 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:19:17.754070 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:19:17.754070 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:19:17.754070 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:19:17.754070 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:19:17.754070 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:19:17.754070 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 14:19:17.754070 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:19:17.754070 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 76/162] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jun 7 14:19:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:19:17.754749 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfbccab-c2d3-469f-b8f4-e7e3e0ce208b None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:17.773366 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:17.791162 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9bf4c6-1cc3-4e6c-8afe-7dd169f3e9f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:17.792612 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9bf4c6-1cc3-4e6c-8afe-7dd169f3e9f5 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:17.792973 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9bf4c6-1cc3-4e6c-8afe-7dd169f3e9f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:17.793327 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9bf4c6-1cc3-4e6c-8afe-7dd169f3e9f5 None admin] PATH_INFO: `/v3/projects/e04d4501005f4310819c6d8307f3df10` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:17.795715 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9bf4c6-1cc3-4e6c-8afe-7dd169f3e9f5 None admin] RBAC: Authorizing `identity:get_project(project_id=e04d4501005f4310819c6d8307f3df10)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:17.797542 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9bf4c6-1cc3-4e6c-8afe-7dd169f3e9f5 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:17.800662 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 87/163] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jun 7 14:19:17 2022] GET /identity/v3/projects/e04d4501005f4310819c6d8307f3df10 => generated 298 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:17.803996 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5784a5f9-c078-4494-812b-f64aa925f654 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:17.803996 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5784a5f9-c078-4494-812b-f64aa925f654 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:17.803996 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5784a5f9-c078-4494-812b-f64aa925f654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:17.824652 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-5784a5f9-c078-4494-812b-f64aa925f654 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:19:17.826454 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-5784a5f9-c078-4494-812b-f64aa925f654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:19:17.890803 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:17.897215 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfbccab-c2d3-469f-b8f4-e7e3e0ce208b None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:17.927182 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 88/164] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:19:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:19:17.928091 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5784a5f9-c078-4494-812b-f64aa925f654 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:17.941614 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e614026f-cab1-492c-bd73-b759f086900b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:17.944434 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e614026f-cab1-492c-bd73-b759f086900b None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:17.944434 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e614026f-cab1-492c-bd73-b759f086900b None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:17.944434 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e614026f-cab1-492c-bd73-b759f086900b None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:17.944434 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e614026f-cab1-492c-bd73-b759f086900b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:19:17.944434 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e614026f-cab1-492c-bd73-b759f086900b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:17.946700 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e614026f-cab1-492c-bd73-b759f086900b None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:17.956856 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 77/165] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jun 7 14:19:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:17.957974 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e614026f-cab1-492c-bd73-b759f086900b None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:18.083868 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e614026f-cab1-492c-bd73-b759f086900b None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:18.129784 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7161dc4d-007a-4a0e-9ea6-82d03db1b6e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:18.130687 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7161dc4d-007a-4a0e-9ea6-82d03db1b6e9 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:18.131111 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7161dc4d-007a-4a0e-9ea6-82d03db1b6e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:18.131533 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7161dc4d-007a-4a0e-9ea6-82d03db1b6e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:18.132737 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7161dc4d-007a-4a0e-9ea6-82d03db1b6e9 None admin] RBAC: Adding query filter params (role.id=9baff2b18f2649f7b757ff2e1f63c853, scope.project.id=e04d4501005f4310819c6d8307f3df10, user.id=3bd6d2364d1d4e71977a1f04d01cd108) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:19:18.133215 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7161dc4d-007a-4a0e-9ea6-82d03db1b6e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:18.135738 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7161dc4d-007a-4a0e-9ea6-82d03db1b6e9 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:18.148444 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5784a5f9-c078-4494-812b-f64aa925f654 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:18.155028 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 78/166] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jun 7 14:19:17 2022] GET /identity/v3/role_assignments?user.id=3bd6d2364d1d4e71977a1f04d01cd108&scope.project.id=e04d4501005f4310819c6d8307f3df10&role.id=9baff2b18f2649f7b757ff2e1f63c853 => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:18.157012 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7161dc4d-007a-4a0e-9ea6-82d03db1b6e9 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:18.184211 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:18.205728 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4975a849-c96a-43fe-8f91-99384f65f600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:18.206555 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4975a849-c96a-43fe-8f91-99384f65f600 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:18.206908 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4975a849-c96a-43fe-8f91-99384f65f600 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:18.207251 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4975a849-c96a-43fe-8f91-99384f65f600 None admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:18.208222 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4975a849-c96a-43fe-8f91-99384f65f600 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:19:18.208577 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4975a849-c96a-43fe-8f91-99384f65f600 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:18.210059 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4975a849-c96a-43fe-8f91-99384f65f600 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:18.216407 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:18.229421 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 89/167] 10.222.0.85 () {56 vars in 1101 bytes} [Tue Jun 7 14:19:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:18.230217 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4975a849-c96a-43fe-8f91-99384f65f600 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:18.428072 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4975a849-c96a-43fe-8f91-99384f65f600 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:18.467852 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58b8cfb-dd15-41f1-9806-a21622eeeb20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:18.468789 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58b8cfb-dd15-41f1-9806-a21622eeeb20 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:18.469077 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58b8cfb-dd15-41f1-9806-a21622eeeb20 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:18.469358 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58b8cfb-dd15-41f1-9806-a21622eeeb20 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:18.482986 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58b8cfb-dd15-41f1-9806-a21622eeeb20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:19:18.486622 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7161dc4d-007a-4a0e-9ea6-82d03db1b6e9 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:18.494267 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-c58b8cfb-dd15-41f1-9806-a21622eeeb20 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 14:19:18.494267 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:19:18.494267 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:19:18.494267 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:19:18.494267 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:19:18.494267 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:19:18.494267 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:19:18.494267 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:19:18.494267 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:19:18.494267 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:19:18.494267 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:19:18.494267 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:19:18.494267 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:19:18.494267 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:19:18.494267 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:19:18.494267 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:19:18.495217 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:19:18.495217 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:19:18.495217 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:19:18.495217 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:19:18.495217 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:19:18.495217 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:19:18.495217 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:19:18.495217 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:19:18.495217 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:19:18.495217 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:19:18.495217 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:19:18.495217 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:19:18.495217 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:19:18.495217 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:19:18.495217 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:19:18.495217 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:19:18.495980 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:19:18.495980 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:19:18.495980 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:19:18.495980 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:19:18.495980 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:19:18.495980 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:19:18.495980 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:19:18.495980 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:19:18.495980 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:19:18.495980 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:19:18.495980 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 14:19:18.495980 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:19:18.495980 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 90/168] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jun 7 14:19:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:19:18.496661 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58b8cfb-dd15-41f1-9806-a21622eeeb20 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:18.549123 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a674712-a04c-42b6-b365-dc197e78a67b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:18.550026 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a674712-a04c-42b6-b365-dc197e78a67b None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:18.550312 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a674712-a04c-42b6-b365-dc197e78a67b None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:18.550593 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a674712-a04c-42b6-b365-dc197e78a67b None admin] PATH_INFO: `/v3/users/37e5547faf8a4aebb8725f07a28cd760` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:18.551892 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a674712-a04c-42b6-b365-dc197e78a67b None admin] RBAC: Authorizing `identity:get_user(user_id=37e5547faf8a4aebb8725f07a28cd760)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:18.553383 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a674712-a04c-42b6-b365-dc197e78a67b None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:18.556483 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 79/169] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jun 7 14:19:18 2022] GET /identity/v3/users/37e5547faf8a4aebb8725f07a28cd760 => generated 275 bytes in 399 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:18.557359 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a674712-a04c-42b6-b365-dc197e78a67b None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:18.558452 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:18.611700 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58b8cfb-dd15-41f1-9806-a21622eeeb20 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:18.650040 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0be9f5e-cea5-4189-9924-707ae72565b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:18.650939 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0be9f5e-cea5-4189-9924-707ae72565b4 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:18.651232 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0be9f5e-cea5-4189-9924-707ae72565b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:18.651519 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0be9f5e-cea5-4189-9924-707ae72565b4 None admin] PATH_INFO: `/v3/users/b228c514ccb9432f8d49131bfd6127be` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:18.652825 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0be9f5e-cea5-4189-9924-707ae72565b4 None admin] RBAC: Authorizing `identity:get_user(user_id=b228c514ccb9432f8d49131bfd6127be)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:18.654425 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0be9f5e-cea5-4189-9924-707ae72565b4 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:18.656221 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 91/170] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jun 7 14:19:18 2022] GET /identity/v3/users/b228c514ccb9432f8d49131bfd6127be => generated 289 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:18.657046 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0be9f5e-cea5-4189-9924-707ae72565b4 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:18.783547 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0be9f5e-cea5-4189-9924-707ae72565b4 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:18.806696 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:18.823222 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4693121c-7648-4b21-a200-0b1788e8a064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:18.824693 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4693121c-7648-4b21-a200-0b1788e8a064 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:18.825130 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4693121c-7648-4b21-a200-0b1788e8a064 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:18.825463 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4693121c-7648-4b21-a200-0b1788e8a064 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:18.837201 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:18.870805 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a674712-a04c-42b6-b365-dc197e78a67b None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:18.918284 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4693121c-7648-4b21-a200-0b1788e8a064 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:18.919789 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4693121c-7648-4b21-a200-0b1788e8a064 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:19.005108 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9226ec6a-4f7f-4f2c-93dc-412c46f14b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:19.005800 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9226ec6a-4f7f-4f2c-93dc-412c46f14b73 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:19.006090 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9226ec6a-4f7f-4f2c-93dc-412c46f14b73 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:19.006368 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9226ec6a-4f7f-4f2c-93dc-412c46f14b73 None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:19.007139 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9226ec6a-4f7f-4f2c-93dc-412c46f14b73 None admin] RBAC: Adding query filter params (name=member) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:19:19.007449 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9226ec6a-4f7f-4f2c-93dc-412c46f14b73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:19.009456 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9226ec6a-4f7f-4f2c-93dc-412c46f14b73 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:19.034604 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 80/171] 10.222.0.85 () {56 vars in 1101 bytes} [Tue Jun 7 14:19:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 477 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:19.035389 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9226ec6a-4f7f-4f2c-93dc-412c46f14b73 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:19.049369 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 92/172] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jun 7 14:19:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 14:19:19.051020 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4693121c-7648-4b21-a200-0b1788e8a064 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:19.076892 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:19.199697 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4693121c-7648-4b21-a200-0b1788e8a064 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:19.264070 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c05fa01-2a12-4e13-87b0-8217b1fc249f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:19.266557 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c05fa01-2a12-4e13-87b0-8217b1fc249f None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:19.268652 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c05fa01-2a12-4e13-87b0-8217b1fc249f None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:19.269741 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c05fa01-2a12-4e13-87b0-8217b1fc249f None admin] PATH_INFO: `/v3/users/37e5547faf8a4aebb8725f07a28cd760` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:19.271413 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c05fa01-2a12-4e13-87b0-8217b1fc249f None admin] RBAC: Authorizing `identity:get_user(user_id=37e5547faf8a4aebb8725f07a28cd760)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:19.277606 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c05fa01-2a12-4e13-87b0-8217b1fc249f None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:19.279629 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 93/173] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jun 7 14:19:19 2022] GET /identity/v3/users/37e5547faf8a4aebb8725f07a28cd760 => generated 275 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:19.280651 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c05fa01-2a12-4e13-87b0-8217b1fc249f None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:19.379815 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:19.403718 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c05fa01-2a12-4e13-87b0-8217b1fc249f None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:19.449457 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9226ec6a-4f7f-4f2c-93dc-412c46f14b73 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:19.469568 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f966935b-9a5d-4fe6-b88a-b39512cb8999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:19.470596 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f966935b-9a5d-4fe6-b88a-b39512cb8999 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:19.471023 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f966935b-9a5d-4fe6-b88a-b39512cb8999 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:19.471386 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f966935b-9a5d-4fe6-b88a-b39512cb8999 None admin] PATH_INFO: `/v3/projects/e04d4501005f4310819c6d8307f3df10` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:19.473780 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f966935b-9a5d-4fe6-b88a-b39512cb8999 None admin] RBAC: Authorizing `identity:get_project(project_id=e04d4501005f4310819c6d8307f3df10)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:19.474781 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f966935b-9a5d-4fe6-b88a-b39512cb8999 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:19.480914 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 94/174] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jun 7 14:19:19 2022] GET /identity/v3/projects/e04d4501005f4310819c6d8307f3df10 => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:19.482088 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f966935b-9a5d-4fe6-b88a-b39512cb8999 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:19.512145 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a7c33d-7a3c-4f38-a9a0-97b64ff067c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:19.515201 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a7c33d-7a3c-4f38-a9a0-97b64ff067c9 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:19.515889 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a7c33d-7a3c-4f38-a9a0-97b64ff067c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:19.516408 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a7c33d-7a3c-4f38-a9a0-97b64ff067c9 None admin] PATH_INFO: `/v3/projects/e04d4501005f4310819c6d8307f3df10` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:19.527869 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a7c33d-7a3c-4f38-a9a0-97b64ff067c9 None admin] RBAC: Authorizing `identity:get_project(project_id=e04d4501005f4310819c6d8307f3df10)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:19.531208 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a7c33d-7a3c-4f38-a9a0-97b64ff067c9 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:19.534781 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 81/175] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jun 7 14:19:19 2022] GET /identity/v3/projects/e04d4501005f4310819c6d8307f3df10 => generated 298 bytes in 500 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:19.545093 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a7c33d-7a3c-4f38-a9a0-97b64ff067c9 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:19.637925 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:19.859413 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f966935b-9a5d-4fe6-b88a-b39512cb8999 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:19.884244 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a7c33d-7a3c-4f38-a9a0-97b64ff067c9 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:19.975808 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e6219f-4d67-44be-8761-c759ba012b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:19.978501 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e6219f-4d67-44be-8761-c759ba012b37 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:19.979180 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e6219f-4d67-44be-8761-c759ba012b37 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:19.980465 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e6219f-4d67-44be-8761-c759ba012b37 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:19.988967 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cce89e-94a7-4ffd-8f1a-a2a916c86935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:19.995653 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cce89e-94a7-4ffd-8f1a-a2a916c86935 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:19.996242 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cce89e-94a7-4ffd-8f1a-a2a916c86935 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:19.996759 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cce89e-94a7-4ffd-8f1a-a2a916c86935 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:19.997814 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cce89e-94a7-4ffd-8f1a-a2a916c86935 None admin] RBAC: Adding query filter params (role.id=494cd000d6884a179c334848f52a3a21, scope.project.id=e04d4501005f4310819c6d8307f3df10, user.id=37e5547faf8a4aebb8725f07a28cd760) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:19:19.998341 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cce89e-94a7-4ffd-8f1a-a2a916c86935 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:20.000071 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cce89e-94a7-4ffd-8f1a-a2a916c86935 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:20.016841 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 82/176] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jun 7 14:19:19 2022] GET /identity/v3/role_assignments?user.id=37e5547faf8a4aebb8725f07a28cd760&scope.project.id=e04d4501005f4310819c6d8307f3df10&role.id=494cd000d6884a179c334848f52a3a21 => generated 608 bytes in 472 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:20.018498 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cce89e-94a7-4ffd-8f1a-a2a916c86935 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:20.026096 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:20.153980 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e6219f-4d67-44be-8761-c759ba012b37 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:20.156402 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e6219f-4d67-44be-8761-c759ba012b37 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:20.221166 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:20.243019 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cce89e-94a7-4ffd-8f1a-a2a916c86935 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:20.286265 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 95/177] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jun 7 14:19:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 805 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 14:19:20.293595 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e6219f-4d67-44be-8761-c759ba012b37 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:20.294530 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953b6732-a994-4f64-a0e1-d52967427da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:20.295459 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953b6732-a994-4f64-a0e1-d52967427da7 None admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:20.295743 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953b6732-a994-4f64-a0e1-d52967427da7 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:20.296021 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953b6732-a994-4f64-a0e1-d52967427da7 None admin] PATH_INFO: `/v3/projects/e04d4501005f4310819c6d8307f3df10/users/b228c514ccb9432f8d49131bfd6127be/roles/e2072a2a35d64eec9d32dac7c47c4419` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:20.298537 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953b6732-a994-4f64-a0e1-d52967427da7 None admin] RBAC: Authorizing `identity:create_grant(project_id=e04d4501005f4310819c6d8307f3df10, user_id=b228c514ccb9432f8d49131bfd6127be, role_id=e2072a2a35d64eec9d32dac7c47c4419)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:20.299669 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953b6732-a994-4f64-a0e1-d52967427da7 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:20.372545 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 83/178] 10.222.0.85 () {58 vars in 1439 bytes} [Tue Jun 7 14:19:20 2022] PUT /identity/v3/projects/e04d4501005f4310819c6d8307f3df10/users/b228c514ccb9432f8d49131bfd6127be/roles/e2072a2a35d64eec9d32dac7c47c4419 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:19:20.374301 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13887792-b87a-4bb5-8c2d-7358aac97292 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:20.374658 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13887792-b87a-4bb5-8c2d-7358aac97292 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:20.374942 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13887792-b87a-4bb5-8c2d-7358aac97292 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:20.377450 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:20.377772 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 84/179] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:19:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:19:20.385121 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685e4e57-09ce-475b-91b4-5c65706f9cb1 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:20.385413 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685e4e57-09ce-475b-91b4-5c65706f9cb1 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:20.385699 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685e4e57-09ce-475b-91b4-5c65706f9cb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:20.423158 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-685e4e57-09ce-475b-91b4-5c65706f9cb1 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:19:20.424809 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-685e4e57-09ce-475b-91b4-5c65706f9cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:19:20.512048 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e6219f-4d67-44be-8761-c759ba012b37 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:20.534061 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 85/180] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:19:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 14:19:20.568486 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d08154-963b-4b6a-9dc0-67532153f60e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:20.569803 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d08154-963b-4b6a-9dc0-67532153f60e None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:20.570263 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d08154-963b-4b6a-9dc0-67532153f60e None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:20.570665 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d08154-963b-4b6a-9dc0-67532153f60e None admin] PATH_INFO: `/v3/projects/eaca1346708e4a3584eada3747bdec8b` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:20.573268 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d08154-963b-4b6a-9dc0-67532153f60e None admin] RBAC: Authorizing `identity:get_project(project_id=eaca1346708e4a3584eada3747bdec8b)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:20.575329 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d08154-963b-4b6a-9dc0-67532153f60e None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:20.582466 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 96/181] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jun 7 14:19:20 2022] GET /identity/v3/projects/eaca1346708e4a3584eada3747bdec8b => generated 312 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:20.583978 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685e4e57-09ce-475b-91b4-5c65706f9cb1 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:20.593992 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d08154-963b-4b6a-9dc0-67532153f60e None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:20.710290 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:20.742126 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685e4e57-09ce-475b-91b4-5c65706f9cb1 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:20.755857 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d08154-963b-4b6a-9dc0-67532153f60e None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:20.782584 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:20.805019 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9851a2ce-8d43-4936-84e0-eba7bd200f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:20.806375 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9851a2ce-8d43-4936-84e0-eba7bd200f26 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:20.806661 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9851a2ce-8d43-4936-84e0-eba7bd200f26 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:20.806937 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9851a2ce-8d43-4936-84e0-eba7bd200f26 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:20.811918 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48098b4d-d8eb-483f-9ad5-a18574bca95c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:20.813002 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48098b4d-d8eb-483f-9ad5-a18574bca95c None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:20.813303 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48098b4d-d8eb-483f-9ad5-a18574bca95c None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:20.813589 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48098b4d-d8eb-483f-9ad5-a18574bca95c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:20.814448 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48098b4d-d8eb-483f-9ad5-a18574bca95c None admin] RBAC: Adding query filter params (role.id=3771341b3be641139e752aab690397a4, scope.project.id=eaca1346708e4a3584eada3747bdec8b, user.id=37e5547faf8a4aebb8725f07a28cd760) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:19:20.814766 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48098b4d-d8eb-483f-9ad5-a18574bca95c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:20.816237 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48098b4d-d8eb-483f-9ad5-a18574bca95c None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:20.833679 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 86/182] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jun 7 14:19:20 2022] GET /identity/v3/role_assignments?user.id=37e5547faf8a4aebb8725f07a28cd760&scope.project.id=eaca1346708e4a3584eada3747bdec8b&role.id=3771341b3be641139e752aab690397a4 => generated 608 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:20.834142 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9851a2ce-8d43-4936-84e0-eba7bd200f26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:19:20.834263 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7164784-8b16-4719-b28d-f8201bd8b2dc None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:20.834263 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7164784-8b16-4719-b28d-f8201bd8b2dc None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:20.834263 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7164784-8b16-4719-b28d-f8201bd8b2dc None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:20.834263 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 87/183] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:19:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:19:20.834263 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872e5fe6-d998-44b5-b4f6-8593ff6a0060 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:20.834732 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-9851a2ce-8d43-4936-84e0-eba7bd200f26 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:19:20.834732 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:19:20.834732 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:19:20.834732 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:19:20.834732 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:19:20.834732 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:19:20.834732 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:19:20.834732 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:19:20.834732 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:19:20.834732 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:19:20.834732 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:19:20.834732 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:19:20.834732 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:19:20.834732 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:19:20.834732 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:19:20.834732 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:19:20.836003 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:19:20.836003 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:19:20.836003 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:19:20.836003 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:19:20.836003 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:19:20.836003 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:19:20.836003 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:19:20.836003 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:19:20.836003 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:19:20.836003 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:19:20.836003 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:19:20.836003 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:19:20.836003 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:19:20.836003 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:19:20.836003 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:19:20.836003 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:19:20.837345 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:19:20.837345 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:19:20.837345 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:19:20.837345 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:19:20.837345 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:19:20.837345 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:19:20.837345 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:19:20.837345 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:19:20.837345 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:19:20.837345 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:19:20.837345 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:19:20.837345 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:19:20.838266 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872e5fe6-d998-44b5-b4f6-8593ff6a0060 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:20.838266 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872e5fe6-d998-44b5-b4f6-8593ff6a0060 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:20.838457 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 97/184] 10.222.0.85 () {56 vars in 1096 bytes} [Tue Jun 7 14:19:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:19:20.839940 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 88/185] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:19:20 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:19:20.841181 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc8ee48-5533-4074-832b-2d42a10fc3a9 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:20.841470 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc8ee48-5533-4074-832b-2d42a10fc3a9 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:20.841639 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9851a2ce-8d43-4936-84e0-eba7bd200f26 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:20.841772 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc8ee48-5533-4074-832b-2d42a10fc3a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:20.867056 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-3bc8ee48-5533-4074-832b-2d42a10fc3a9 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:19:20.869391 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-3bc8ee48-5533-4074-832b-2d42a10fc3a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:19:20.983202 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 89/186] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:19:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:19:20.985470 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b2564-a329-4cd0-b81a-fdc70ba3d5d7 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:20.985974 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b2564-a329-4cd0-b81a-fdc70ba3d5d7 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:20.986429 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b2564-a329-4cd0-b81a-fdc70ba3d5d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:21.005988 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9851a2ce-8d43-4936-84e0-eba7bd200f26 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:21.009734 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-d08b2564-a329-4cd0-b81a-fdc70ba3d5d7 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:19:21.012322 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-d08b2564-a329-4cd0-b81a-fdc70ba3d5d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:19:21.079604 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eb3591-c9bd-4086-8b55-3514d82bbc87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:21.080775 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eb3591-c9bd-4086-8b55-3514d82bbc87 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:21.081331 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eb3591-c9bd-4086-8b55-3514d82bbc87 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:21.081877 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eb3591-c9bd-4086-8b55-3514d82bbc87 None admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:21.083358 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eb3591-c9bd-4086-8b55-3514d82bbc87 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:19:21.083963 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eb3591-c9bd-4086-8b55-3514d82bbc87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:21.086915 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eb3591-c9bd-4086-8b55-3514d82bbc87 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:21.106755 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 98/187] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jun 7 14:19:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:19:21.108335 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eb3591-c9bd-4086-8b55-3514d82bbc87 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:21.121631 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:21.127295 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 90/188] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:19:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:19:21.128237 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08b2564-a329-4cd0-b81a-fdc70ba3d5d7 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:21.173970 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:21.280643 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08b2564-a329-4cd0-b81a-fdc70ba3d5d7 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:21.292737 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eb3591-c9bd-4086-8b55-3514d82bbc87 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:21.325702 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca82bd86-a19a-44c1-8156-49ee6dcb981e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:21.326774 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca82bd86-a19a-44c1-8156-49ee6dcb981e None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:21.327414 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca82bd86-a19a-44c1-8156-49ee6dcb981e None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:21.327700 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca82bd86-a19a-44c1-8156-49ee6dcb981e None admin] PATH_INFO: `/v3/users/026b6f0bce8f4978919659f227dca61b` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:21.343379 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0101855c-5011-4d02-ac29-06caa04ad216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:21.343592 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca82bd86-a19a-44c1-8156-49ee6dcb981e None admin] RBAC: Authorizing `identity:get_user(user_id=026b6f0bce8f4978919659f227dca61b)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:21.344125 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0101855c-5011-4d02-ac29-06caa04ad216 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:21.344440 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0101855c-5011-4d02-ac29-06caa04ad216 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:21.344726 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0101855c-5011-4d02-ac29-06caa04ad216 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:21.344968 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca82bd86-a19a-44c1-8156-49ee6dcb981e None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:21.347121 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 91/189] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jun 7 14:19:21 2022] GET /identity/v3/users/026b6f0bce8f4978919659f227dca61b => generated 283 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:21.347801 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca82bd86-a19a-44c1-8156-49ee6dcb981e None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:21.363132 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0101855c-5011-4d02-ac29-06caa04ad216 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:19:21.372512 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-0101855c-5011-4d02-ac29-06caa04ad216 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 14:19:21.372512 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:19:21.372512 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:19:21.372512 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:19:21.372512 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:19:21.372512 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:19:21.372512 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:19:21.372512 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:19:21.372512 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:19:21.372512 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:19:21.372512 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:19:21.372512 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:19:21.372512 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:19:21.372512 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:19:21.372512 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:19:21.372512 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:19:21.373588 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:19:21.373588 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:19:21.373588 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:19:21.373588 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:19:21.373588 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:19:21.373588 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:19:21.373588 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:19:21.373588 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:19:21.373588 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:19:21.373588 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:19:21.373588 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:19:21.373588 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:19:21.373588 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:19:21.373588 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:19:21.373588 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:19:21.373588 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:19:21.374401 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:19:21.374401 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:19:21.374401 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:19:21.374401 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:19:21.374401 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:19:21.374401 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:19:21.374401 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:19:21.374401 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:19:21.374401 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:19:21.374401 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:19:21.374401 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 14:19:21.374401 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:19:21.374401 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 99/190] 10.222.0.85 () {56 vars in 1114 bytes} [Tue Jun 7 14:19:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:19:21.375052 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0101855c-5011-4d02-ac29-06caa04ad216 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:21.418808 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:21.474880 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca82bd86-a19a-44c1-8156-49ee6dcb981e None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:21.512488 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:21.514994 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2851722-f31e-4744-8775-e642b5986d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:21.515865 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2851722-f31e-4744-8775-e642b5986d16 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:21.516119 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2851722-f31e-4744-8775-e642b5986d16 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:21.516630 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2851722-f31e-4744-8775-e642b5986d16 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:21.516917 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0101855c-5011-4d02-ac29-06caa04ad216 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:21.517858 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2851722-f31e-4744-8775-e642b5986d16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:21.519385 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2851722-f31e-4744-8775-e642b5986d16 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:21.524659 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 92/191] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:19:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:21.524659 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2851722-f31e-4744-8775-e642b5986d16 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:21.572142 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aff0d4-6694-4d78-baad-23c8b86cddc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:21.572880 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aff0d4-6694-4d78-baad-23c8b86cddc8 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:21.573170 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aff0d4-6694-4d78-baad-23c8b86cddc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:21.573454 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aff0d4-6694-4d78-baad-23c8b86cddc8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:21.674349 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2851722-f31e-4744-8775-e642b5986d16 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:21.704341 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98aff0d4-6694-4d78-baad-23c8b86cddc8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:21.707245 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98aff0d4-6694-4d78-baad-23c8b86cddc8 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:21.713581 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:21.726208 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb56893-1e9b-426a-9106-d60508956813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:21.727291 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb56893-1e9b-426a-9106-d60508956813 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:21.727782 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb56893-1e9b-426a-9106-d60508956813 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:21.728282 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb56893-1e9b-426a-9106-d60508956813 None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:21.729402 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb56893-1e9b-426a-9106-d60508956813 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:19:21.729792 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb56893-1e9b-426a-9106-d60508956813 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:21.732139 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb56893-1e9b-426a-9106-d60508956813 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:21.751704 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 93/192] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jun 7 14:19:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:21.751704 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb56893-1e9b-426a-9106-d60508956813 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:21.850491 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:21.857105 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 100/193] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jun 7 14:19:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 14:19:21.864213 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aff0d4-6694-4d78-baad-23c8b86cddc8 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:21.893905 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb56893-1e9b-426a-9106-d60508956813 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:21.939780 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3fa9fc-ea83-4992-ba87-351b357d7d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:21.940045 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3fa9fc-ea83-4992-ba87-351b357d7d96 None admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:21.940356 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3fa9fc-ea83-4992-ba87-351b357d7d96 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:21.940848 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3fa9fc-ea83-4992-ba87-351b357d7d96 None admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:21.941327 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3fa9fc-ea83-4992-ba87-351b357d7d96 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:21.942820 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3fa9fc-ea83-4992-ba87-351b357d7d96 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:22.014580 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aff0d4-6694-4d78-baad-23c8b86cddc8 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:22.051990 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21cb5e0-a32f-4bf7-94fc-0a279a99d6c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:22.053014 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21cb5e0-a32f-4bf7-94fc-0a279a99d6c3 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:22.053313 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21cb5e0-a32f-4bf7-94fc-0a279a99d6c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:22.053701 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21cb5e0-a32f-4bf7-94fc-0a279a99d6c3 None admin] PATH_INFO: `/v3/users/026b6f0bce8f4978919659f227dca61b` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:22.054871 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21cb5e0-a32f-4bf7-94fc-0a279a99d6c3 None admin] RBAC: Authorizing `identity:get_user(user_id=026b6f0bce8f4978919659f227dca61b)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:22.056409 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21cb5e0-a32f-4bf7-94fc-0a279a99d6c3 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:22.062698 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 101/194] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jun 7 14:19:21 2022] GET /identity/v3/users/026b6f0bce8f4978919659f227dca61b => generated 283 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:22.062698 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21cb5e0-a32f-4bf7-94fc-0a279a99d6c3 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:22.069273 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 94/195] 10.222.0.85 () {60 vars in 1132 bytes} [Tue Jun 7 14:19:21 2022] POST /identity/v3/users => generated 297 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:19:22.070753 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3fa9fc-ea83-4992-ba87-351b357d7d96 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:22.174050 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:22.186034 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:22.238202 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21cb5e0-a32f-4bf7-94fc-0a279a99d6c3 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:22.319194 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3fa9fc-ea83-4992-ba87-351b357d7d96 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:22.327243 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac039fa-39a2-4efd-933b-c10e1016424d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:22.329700 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac039fa-39a2-4efd-933b-c10e1016424d None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:22.330443 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac039fa-39a2-4efd-933b-c10e1016424d None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:22.331202 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac039fa-39a2-4efd-933b-c10e1016424d None admin] PATH_INFO: `/v3/projects/f95de357ccad493984042656b2c0689a` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:22.333312 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac039fa-39a2-4efd-933b-c10e1016424d None admin] RBAC: Authorizing `identity:get_project(project_id=f95de357ccad493984042656b2c0689a)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:22.337673 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac039fa-39a2-4efd-933b-c10e1016424d None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:22.345027 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 102/196] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jun 7 14:19:22 2022] GET /identity/v3/projects/f95de357ccad493984042656b2c0689a => generated 302 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:22.346577 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac039fa-39a2-4efd-933b-c10e1016424d None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:22.373851 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f7c9bc-4e28-4b0a-bfb6-801832c927b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:22.376546 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f7c9bc-4e28-4b0a-bfb6-801832c927b9 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:22.377744 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f7c9bc-4e28-4b0a-bfb6-801832c927b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:22.378551 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f7c9bc-4e28-4b0a-bfb6-801832c927b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:22.486304 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f7c9bc-4e28-4b0a-bfb6-801832c927b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:22.487885 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f7c9bc-4e28-4b0a-bfb6-801832c927b9 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:22.501049 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac039fa-39a2-4efd-933b-c10e1016424d None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:22.552517 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa497dd6-159e-43b1-996f-f75e5bdc7705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:22.553817 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa497dd6-159e-43b1-996f-f75e5bdc7705 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:22.554702 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa497dd6-159e-43b1-996f-f75e5bdc7705 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:22.555217 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa497dd6-159e-43b1-996f-f75e5bdc7705 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:22.558023 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa497dd6-159e-43b1-996f-f75e5bdc7705 None admin] RBAC: Adding query filter params (role.id=9baff2b18f2649f7b757ff2e1f63c853, scope.project.id=f95de357ccad493984042656b2c0689a, user.id=026b6f0bce8f4978919659f227dca61b) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:19:22.558644 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa497dd6-159e-43b1-996f-f75e5bdc7705 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:22.560447 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa497dd6-159e-43b1-996f-f75e5bdc7705 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:22.566637 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:22.569649 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:22.572211 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 103/197] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jun 7 14:19:22 2022] GET /identity/v3/role_assignments?user.id=026b6f0bce8f4978919659f227dca61b&scope.project.id=f95de357ccad493984042656b2c0689a&role.id=9baff2b18f2649f7b757ff2e1f63c853 => generated 259 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:19:22.679544 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 95/198] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jun 7 14:19:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 610 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 14:19:22.686967 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa497dd6-159e-43b1-996f-f75e5bdc7705 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:22.849313 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa497dd6-159e-43b1-996f-f75e5bdc7705 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:22.890561 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce776b4-c6a6-4325-8c29-ade254b22fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:22.891490 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce776b4-c6a6-4325-8c29-ade254b22fe8 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:22.891778 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce776b4-c6a6-4325-8c29-ade254b22fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:22.892110 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce776b4-c6a6-4325-8c29-ade254b22fe8 None admin] PATH_INFO: `/v3/projects/f95de357ccad493984042656b2c0689a` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:22.893383 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce776b4-c6a6-4325-8c29-ade254b22fe8 None admin] RBAC: Authorizing `identity:get_project(project_id=f95de357ccad493984042656b2c0689a)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:22.894886 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce776b4-c6a6-4325-8c29-ade254b22fe8 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:22.896879 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 104/199] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jun 7 14:19:22 2022] GET /identity/v3/projects/f95de357ccad493984042656b2c0689a => generated 302 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:22.903374 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f7c9bc-4e28-4b0a-bfb6-801832c927b9 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:23.052385 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f7c9bc-4e28-4b0a-bfb6-801832c927b9 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:23.079975 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8177de4-db50-4379-8735-668ec108ae35 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:23.080691 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8177de4-db50-4379-8735-668ec108ae35 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:23.081333 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8177de4-db50-4379-8735-668ec108ae35 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:23.082951 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 105/200] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:19:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:19:23.090029 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf437fb3-f840-4bad-9891-54399a1f2801 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:23.090807 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf437fb3-f840-4bad-9891-54399a1f2801 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:23.093599 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf437fb3-f840-4bad-9891-54399a1f2801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:23.120476 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af8e782-275b-43b6-a322-7cd830f15388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:23.120476 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af8e782-275b-43b6-a322-7cd830f15388 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:23.120476 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af8e782-275b-43b6-a322-7cd830f15388 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:23.120476 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af8e782-275b-43b6-a322-7cd830f15388 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:23.121675 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af8e782-275b-43b6-a322-7cd830f15388 None admin] RBAC: Adding query filter params (role.id=494cd000d6884a179c334848f52a3a21, scope.project.id=f95de357ccad493984042656b2c0689a, user.id=026b6f0bce8f4978919659f227dca61b) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:19:23.122168 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af8e782-275b-43b6-a322-7cd830f15388 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:23.124516 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af8e782-275b-43b6-a322-7cd830f15388 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:23.125734 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-bf437fb3-f840-4bad-9891-54399a1f2801 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:19:23.126934 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-bf437fb3-f840-4bad-9891-54399a1f2801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:19:23.140050 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 96/201] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jun 7 14:19:22 2022] GET /identity/v3/role_assignments?user.id=026b6f0bce8f4978919659f227dca61b&scope.project.id=f95de357ccad493984042656b2c0689a&role.id=494cd000d6884a179c334848f52a3a21 => generated 259 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:23.221026 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:23.237979 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 106/202] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:19:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:19:23.276174 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af8e782-275b-43b6-a322-7cd830f15388 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:23.286014 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:23.410947 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af8e782-275b-43b6-a322-7cd830f15388 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:23.465305 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc897094-c5b0-4d75-a815-69385d2ef023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:23.467309 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc897094-c5b0-4d75-a815-69385d2ef023 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:23.468030 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc897094-c5b0-4d75-a815-69385d2ef023 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:23.468841 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc897094-c5b0-4d75-a815-69385d2ef023 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:23.492265 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc897094-c5b0-4d75-a815-69385d2ef023 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:19:23.501327 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-dc897094-c5b0-4d75-a815-69385d2ef023 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 14:19:23.501327 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:19:23.501327 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:19:23.501327 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:19:23.501327 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:19:23.501327 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:19:23.501327 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:19:23.501327 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:19:23.501327 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:19:23.501327 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:19:23.501327 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:19:23.501327 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:19:23.501327 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:19:23.501327 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:19:23.501327 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:19:23.501327 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:19:23.502261 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:19:23.502261 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:19:23.502261 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:19:23.502261 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:19:23.502261 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:19:23.502261 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:19:23.502261 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:19:23.502261 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:19:23.502261 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:19:23.502261 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:19:23.502261 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:19:23.502261 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:19:23.502261 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:19:23.502261 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:19:23.502261 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:19:23.502261 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:19:23.503048 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:19:23.503048 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:19:23.503048 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:19:23.503048 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:19:23.503048 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:19:23.503048 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:19:23.503048 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:19:23.503048 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:19:23.503048 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:19:23.503048 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:19:23.503048 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 14:19:23.503048 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:19:23.505790 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 97/203] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jun 7 14:19:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:19:23.514318 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf437fb3-f840-4bad-9891-54399a1f2801 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:23.637555 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf437fb3-f840-4bad-9891-54399a1f2801 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:23.706380 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2f211c-efa0-4699-b0c5-737af22aba86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:23.707488 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2f211c-efa0-4699-b0c5-737af22aba86 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:23.707976 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2f211c-efa0-4699-b0c5-737af22aba86 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:23.708479 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2f211c-efa0-4699-b0c5-737af22aba86 None admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:23.709450 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2f211c-efa0-4699-b0c5-737af22aba86 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:19:23.709976 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2f211c-efa0-4699-b0c5-737af22aba86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:23.711631 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2f211c-efa0-4699-b0c5-737af22aba86 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:23.726147 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 107/204] 10.222.0.85 () {56 vars in 1101 bytes} [Tue Jun 7 14:19:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:23.732745 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc897094-c5b0-4d75-a815-69385d2ef023 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:23.864028 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:23.983316 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc897094-c5b0-4d75-a815-69385d2ef023 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:24.078838 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe184b6-5dad-4565-b983-2f0761ff915a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:24.081189 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe184b6-5dad-4565-b983-2f0761ff915a None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:24.082303 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe184b6-5dad-4565-b983-2f0761ff915a None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:24.083315 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe184b6-5dad-4565-b983-2f0761ff915a None admin] PATH_INFO: `/v3/users/b228c514ccb9432f8d49131bfd6127be` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:24.086617 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe184b6-5dad-4565-b983-2f0761ff915a None admin] RBAC: Authorizing `identity:get_user(user_id=b228c514ccb9432f8d49131bfd6127be)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:24.090191 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe184b6-5dad-4565-b983-2f0761ff915a None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:24.094155 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 98/205] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jun 7 14:19:23 2022] GET /identity/v3/users/b228c514ccb9432f8d49131bfd6127be => generated 289 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:24.105589 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2f211c-efa0-4699-b0c5-737af22aba86 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:24.205113 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:24.260017 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2f211c-efa0-4699-b0c5-737af22aba86 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:24.311471 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df67c3e3-2dda-4997-916b-70068d482f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:24.312130 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df67c3e3-2dda-4997-916b-70068d482f01 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:24.312449 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df67c3e3-2dda-4997-916b-70068d482f01 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:24.312732 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df67c3e3-2dda-4997-916b-70068d482f01 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:24.410902 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df67c3e3-2dda-4997-916b-70068d482f01 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:24.412389 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df67c3e3-2dda-4997-916b-70068d482f01 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:24.528861 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:24.629245 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 108/206] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jun 7 14:19:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 523 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 14:19:24.637012 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe184b6-5dad-4565-b983-2f0761ff915a None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:24.823965 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe184b6-5dad-4565-b983-2f0761ff915a None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:24.906611 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951d2437-038c-446b-b9ac-3d08838cbb2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:24.908513 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951d2437-038c-446b-b9ac-3d08838cbb2d None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:24.909170 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951d2437-038c-446b-b9ac-3d08838cbb2d None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:24.909880 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951d2437-038c-446b-b9ac-3d08838cbb2d None admin] PATH_INFO: `/v3/projects/e04d4501005f4310819c6d8307f3df10` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:24.911861 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951d2437-038c-446b-b9ac-3d08838cbb2d None admin] RBAC: Authorizing `identity:get_project(project_id=e04d4501005f4310819c6d8307f3df10)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:24.914395 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951d2437-038c-446b-b9ac-3d08838cbb2d None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:24.918605 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 99/207] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jun 7 14:19:24 2022] GET /identity/v3/projects/e04d4501005f4310819c6d8307f3df10 => generated 298 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:24.924188 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df67c3e3-2dda-4997-916b-70068d482f01 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:25.076434 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df67c3e3-2dda-4997-916b-70068d482f01 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:25.120517 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868b1051-ee94-43ee-96b1-78ac8724626d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:25.121199 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868b1051-ee94-43ee-96b1-78ac8724626d None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:25.121482 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868b1051-ee94-43ee-96b1-78ac8724626d None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:25.121758 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868b1051-ee94-43ee-96b1-78ac8724626d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:25.122616 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868b1051-ee94-43ee-96b1-78ac8724626d None admin] RBAC: Adding query filter params (role.id=e2072a2a35d64eec9d32dac7c47c4419, scope.project.id=e04d4501005f4310819c6d8307f3df10, user.id=b228c514ccb9432f8d49131bfd6127be) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:19:25.122926 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868b1051-ee94-43ee-96b1-78ac8724626d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:25.124625 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868b1051-ee94-43ee-96b1-78ac8724626d None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:25.131790 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 109/208] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jun 7 14:19:24 2022] GET /identity/v3/role_assignments?user.id=b228c514ccb9432f8d49131bfd6127be&scope.project.id=e04d4501005f4310819c6d8307f3df10&role.id=e2072a2a35d64eec9d32dac7c47c4419 => generated 608 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:25.555867 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbfd7b3-0783-43d1-a516-93a84904dfe1 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:25.556813 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbfd7b3-0783-43d1-a516-93a84904dfe1 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:25.557479 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbfd7b3-0783-43d1-a516-93a84904dfe1 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:25.559807 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 100/209] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:19:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:19:25.567761 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f660ff9c-9d4e-4310-82f0-71b1b5f16c20 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:25.569459 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f660ff9c-9d4e-4310-82f0-71b1b5f16c20 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:25.570230 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f660ff9c-9d4e-4310-82f0-71b1b5f16c20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:25.597099 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-f660ff9c-9d4e-4310-82f0-71b1b5f16c20 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:19:25.598635 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-f660ff9c-9d4e-4310-82f0-71b1b5f16c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:19:25.621808 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:25.679191 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 110/210] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:19:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:19:25.704193 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbfd7b3-0783-43d1-a516-93a84904dfe1 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:25.707651 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:25.792320 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d453d-f6ea-41f4-89fa-e49d8ad87328 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:25.792686 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d453d-f6ea-41f4-89fa-e49d8ad87328 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:25.792933 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d453d-f6ea-41f4-89fa-e49d8ad87328 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:25.794541 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 111/211] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:19:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:19:25.800901 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cf9d16-2a80-41fc-8757-8e7e8332a884 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:25.801195 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cf9d16-2a80-41fc-8757-8e7e8332a884 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:25.801467 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cf9d16-2a80-41fc-8757-8e7e8332a884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:25.821521 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-00cf9d16-2a80-41fc-8757-8e7e8332a884 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:19:25.822623 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-00cf9d16-2a80-41fc-8757-8e7e8332a884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:19:25.837172 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbfd7b3-0783-43d1-a516-93a84904dfe1 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:25.874810 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aa0f35-0d5a-4c2c-a653-8d46615f7349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:25.875412 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aa0f35-0d5a-4c2c-a653-8d46615f7349 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:25.875693 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aa0f35-0d5a-4c2c-a653-8d46615f7349 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:25.875971 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aa0f35-0d5a-4c2c-a653-8d46615f7349 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:25.889427 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0aa0f35-0d5a-4c2c-a653-8d46615f7349 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:19:25.898588 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-c0aa0f35-0d5a-4c2c-a653-8d46615f7349 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 14:19:25.898588 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:19:25.898588 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:19:25.898588 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:19:25.898588 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:19:25.898588 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:19:25.898588 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:19:25.898588 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:19:25.898588 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:19:25.898588 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:19:25.898588 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:19:25.898588 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:19:25.898588 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:19:25.898588 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:19:25.898588 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:19:25.898588 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:19:25.899605 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:19:25.899605 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:19:25.899605 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:19:25.899605 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:19:25.899605 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:19:25.899605 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:19:25.899605 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:19:25.899605 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:19:25.899605 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:19:25.899605 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:19:25.899605 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:19:25.899605 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:19:25.899605 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:19:25.899605 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:19:25.899605 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:19:25.899605 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:19:25.900490 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:19:25.900490 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:19:25.900490 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:19:25.900490 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:19:25.900490 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:19:25.900490 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:19:25.900490 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:19:25.900490 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:19:25.900490 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:19:25.900490 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:19:25.900490 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 14:19:25.900490 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:19:25.900490 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 101/212] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jun 7 14:19:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:19:25.905933 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aa0f35-0d5a-4c2c-a653-8d46615f7349 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:25.908841 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 112/213] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:19:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 14:19:25.935582 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cf9d16-2a80-41fc-8757-8e7e8332a884 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:25.987668 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:26.020142 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aa0f35-0d5a-4c2c-a653-8d46615f7349 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:26.027116 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:26.048043 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cf9d16-2a80-41fc-8757-8e7e8332a884 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:26.059819 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b407b5e5-3afa-4d93-8319-cad680a51456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:26.060848 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b407b5e5-3afa-4d93-8319-cad680a51456 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:26.061290 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b407b5e5-3afa-4d93-8319-cad680a51456 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:26.061754 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b407b5e5-3afa-4d93-8319-cad680a51456 None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:26.062683 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b407b5e5-3afa-4d93-8319-cad680a51456 None admin] RBAC: Adding query filter params (name=member) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:19:26.063173 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b407b5e5-3afa-4d93-8319-cad680a51456 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:26.064926 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b407b5e5-3afa-4d93-8319-cad680a51456 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:26.075752 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 102/214] 10.222.0.85 () {56 vars in 1101 bytes} [Tue Jun 7 14:19:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:26.081656 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b407b5e5-3afa-4d93-8319-cad680a51456 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:26.087079 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c38b5c1-def8-4131-8ecf-67d9aa824767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:26.087885 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c38b5c1-def8-4131-8ecf-67d9aa824767 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:26.088176 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c38b5c1-def8-4131-8ecf-67d9aa824767 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:26.088476 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c38b5c1-def8-4131-8ecf-67d9aa824767 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:26.089733 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c38b5c1-def8-4131-8ecf-67d9aa824767 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:26.091317 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c38b5c1-def8-4131-8ecf-67d9aa824767 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:26.097552 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 113/215] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:19:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:26.098565 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c38b5c1-def8-4131-8ecf-67d9aa824767 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:26.210937 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b407b5e5-3afa-4d93-8319-cad680a51456 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:26.222529 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c38b5c1-def8-4131-8ecf-67d9aa824767 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:26.240770 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:26.253060 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3084129-5f70-4de0-98e2-6d059215a2d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:26.253947 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3084129-5f70-4de0-98e2-6d059215a2d0 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:26.254450 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3084129-5f70-4de0-98e2-6d059215a2d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:26.254558 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3084129-5f70-4de0-98e2-6d059215a2d0 None admin] PATH_INFO: `/v3/users/5a235d1390ea42c9b56f16bffef3d4e0` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:26.262910 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9d779a-de78-4908-8232-990af7df2b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:26.263636 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9d779a-de78-4908-8232-990af7df2b73 None admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:26.263991 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9d779a-de78-4908-8232-990af7df2b73 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:26.264240 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9d779a-de78-4908-8232-990af7df2b73 None admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:26.264943 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9d779a-de78-4908-8232-990af7df2b73 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:26.266379 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9d779a-de78-4908-8232-990af7df2b73 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:26.268330 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3084129-5f70-4de0-98e2-6d059215a2d0 None admin] RBAC: Authorizing `identity:get_user(user_id=5a235d1390ea42c9b56f16bffef3d4e0)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:26.269812 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3084129-5f70-4de0-98e2-6d059215a2d0 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:26.271742 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 103/216] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jun 7 14:19:26 2022] GET /identity/v3/users/5a235d1390ea42c9b56f16bffef3d4e0 => generated 297 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:19:26.273981 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94097f26-b648-4cd4-b890-8bbb3e7e11ad None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:26.274934 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94097f26-b648-4cd4-b890-8bbb3e7e11ad None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:26.275788 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94097f26-b648-4cd4-b890-8bbb3e7e11ad None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:26.278040 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 104/217] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:19:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:19:26.279079 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94097f26-b648-4cd4-b890-8bbb3e7e11ad None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:26.379252 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94097f26-b648-4cd4-b890-8bbb3e7e11ad None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:26.404404 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 114/218] 10.222.0.85 () {60 vars in 1132 bytes} [Tue Jun 7 14:19:26 2022] POST /identity/v3/users => generated 297 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:19:26.404404 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26305969-21df-4b39-b4d2-aeca75c1c248 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:26.404404 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26305969-21df-4b39-b4d2-aeca75c1c248 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:26.404404 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26305969-21df-4b39-b4d2-aeca75c1c248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:26.407847 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:26.415858 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-26305969-21df-4b39-b4d2-aeca75c1c248 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:19:26.418246 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-26305969-21df-4b39-b4d2-aeca75c1c248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:19:26.420565 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc30840d-9d72-46cc-8b8c-f4f56cd2de45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:26.421253 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc30840d-9d72-46cc-8b8c-f4f56cd2de45 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:26.421596 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc30840d-9d72-46cc-8b8c-f4f56cd2de45 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:26.421884 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc30840d-9d72-46cc-8b8c-f4f56cd2de45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:26.485193 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:26.510665 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc30840d-9d72-46cc-8b8c-f4f56cd2de45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:26.512294 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc30840d-9d72-46cc-8b8c-f4f56cd2de45 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:26.522370 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 115/219] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:19:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:19:26.524931 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ae7c87-1b97-4beb-95ba-5c705eb08872 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:26.525210 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ae7c87-1b97-4beb-95ba-5c705eb08872 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:26.525487 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ae7c87-1b97-4beb-95ba-5c705eb08872 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:26.528408 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 116/220] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:19:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:19:26.532776 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982f6305-eaa3-4bfb-acff-1012488a0516 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:26.533066 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982f6305-eaa3-4bfb-acff-1012488a0516 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:26.533458 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982f6305-eaa3-4bfb-acff-1012488a0516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:26.558975 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-982f6305-eaa3-4bfb-acff-1012488a0516 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:19:26.560952 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-982f6305-eaa3-4bfb-acff-1012488a0516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:19:26.640747 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 117/221] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:19:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 14:19:26.648498 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982f6305-eaa3-4bfb-acff-1012488a0516 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:26.653802 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 105/222] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jun 7 14:19:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 14:19:26.659248 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc30840d-9d72-46cc-8b8c-f4f56cd2de45 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:26.743829 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:26.767226 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:26.789237 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982f6305-eaa3-4bfb-acff-1012488a0516 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:26.794463 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc30840d-9d72-46cc-8b8c-f4f56cd2de45 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:26.832744 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532aba56-ddff-45a2-a105-e65ffede4f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:26.833572 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532aba56-ddff-45a2-a105-e65ffede4f61 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:26.833872 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532aba56-ddff-45a2-a105-e65ffede4f61 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:26.834355 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532aba56-ddff-45a2-a105-e65ffede4f61 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:26.841347 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db06c0ce-5c45-4fcd-bd19-664904794c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:26.842791 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db06c0ce-5c45-4fcd-bd19-664904794c1f None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:26.843274 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db06c0ce-5c45-4fcd-bd19-664904794c1f None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:26.844158 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db06c0ce-5c45-4fcd-bd19-664904794c1f None admin] PATH_INFO: `/v3/projects/f95de357ccad493984042656b2c0689a` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:26.845692 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db06c0ce-5c45-4fcd-bd19-664904794c1f None admin] RBAC: Authorizing `identity:get_project(project_id=f95de357ccad493984042656b2c0689a)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:26.846750 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532aba56-ddff-45a2-a105-e65ffede4f61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:19:26.847892 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db06c0ce-5c45-4fcd-bd19-664904794c1f None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:26.854662 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 106/223] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jun 7 14:19:26 2022] GET /identity/v3/projects/f95de357ccad493984042656b2c0689a => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:19:26.855351 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-532aba56-ddff-45a2-a105-e65ffede4f61 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 14:19:26.855351 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:19:26.855351 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:19:26.855351 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:19:26.855351 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:19:26.855351 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:19:26.855351 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:19:26.855351 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:19:26.855351 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:19:26.855351 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:19:26.855351 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:19:26.855351 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:19:26.855351 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:19:26.855351 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:19:26.855351 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:19:26.855351 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:19:26.856202 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:19:26.856202 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:19:26.856202 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:19:26.856202 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:19:26.856202 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:19:26.856202 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:19:26.856202 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:19:26.856202 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:19:26.856202 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:19:26.856202 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:19:26.856202 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:19:26.856202 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:19:26.856202 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:19:26.856202 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:19:26.856202 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:19:26.856202 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:19:26.857438 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:19:26.857438 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:19:26.857438 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:19:26.857438 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:19:26.857438 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:19:26.857438 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:19:26.857438 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:19:26.857438 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:19:26.857438 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:19:26.857438 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:19:26.857438 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 14:19:26.857438 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:19:26.857438 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 118/224] 10.222.0.85 () {56 vars in 1114 bytes} [Tue Jun 7 14:19:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:19:26.858969 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db06c0ce-5c45-4fcd-bd19-664904794c1f None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:26.861285 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532aba56-ddff-45a2-a105-e65ffede4f61 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:26.996258 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db06c0ce-5c45-4fcd-bd19-664904794c1f None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:27.002154 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532aba56-ddff-45a2-a105-e65ffede4f61 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:27.032955 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:27.042138 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c33ed3b-a04f-4ed1-b788-e173deb7f0fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:27.053246 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c33ed3b-a04f-4ed1-b788-e173deb7f0fb None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:27.053246 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c33ed3b-a04f-4ed1-b788-e173deb7f0fb None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:27.053246 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c33ed3b-a04f-4ed1-b788-e173deb7f0fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:27.053246 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c33ed3b-a04f-4ed1-b788-e173deb7f0fb None admin] RBAC: Adding query filter params (role.id=3771341b3be641139e752aab690397a4, scope.project.id=f95de357ccad493984042656b2c0689a, user.id=5a235d1390ea42c9b56f16bffef3d4e0) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:19:27.053246 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c33ed3b-a04f-4ed1-b788-e173deb7f0fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:27.053760 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e23521-bf06-4397-adaf-9cfc1e3c3303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:27.053866 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c33ed3b-a04f-4ed1-b788-e173deb7f0fb None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:27.053935 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e23521-bf06-4397-adaf-9cfc1e3c3303 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:27.053935 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e23521-bf06-4397-adaf-9cfc1e3c3303 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:27.053935 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e23521-bf06-4397-adaf-9cfc1e3c3303 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:27.055320 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 119/225] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jun 7 14:19:26 2022] GET /identity/v3/role_assignments?user.id=5a235d1390ea42c9b56f16bffef3d4e0&scope.project.id=f95de357ccad493984042656b2c0689a&role.id=3771341b3be641139e752aab690397a4 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:27.056645 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c33ed3b-a04f-4ed1-b788-e173deb7f0fb None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:27.081646 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e23521-bf06-4397-adaf-9cfc1e3c3303 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:19:27.091132 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-28e23521-bf06-4397-adaf-9cfc1e3c3303 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:19:27.091132 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:19:27.091132 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:19:27.091132 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:19:27.091132 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:19:27.091132 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:19:27.091132 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:19:27.091132 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:19:27.091132 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:19:27.091132 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:19:27.091132 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:19:27.091132 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:19:27.091132 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:19:27.091132 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:19:27.091132 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:19:27.091132 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:19:27.092683 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:19:27.092683 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:19:27.092683 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:19:27.092683 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:19:27.092683 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:19:27.092683 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:19:27.092683 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:19:27.092683 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:19:27.092683 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:19:27.092683 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:19:27.092683 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:19:27.092683 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:19:27.092683 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:19:27.092683 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:19:27.092683 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:19:27.092683 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:19:27.093740 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:19:27.093740 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:19:27.093740 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:19:27.093740 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:19:27.093740 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:19:27.093740 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:19:27.093740 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:19:27.093740 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:19:27.093740 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:19:27.093740 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:19:27.093740 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:19:27.093740 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:19:27.093740 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 107/226] 10.222.0.85 () {56 vars in 1096 bytes} [Tue Jun 7 14:19:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:19:27.108826 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e23521-bf06-4397-adaf-9cfc1e3c3303 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:27.149489 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:27.371726 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c33ed3b-a04f-4ed1-b788-e173deb7f0fb None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:27.413454 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e23521-bf06-4397-adaf-9cfc1e3c3303 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:27.456523 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05affe1b-95f9-4a70-8f76-01b12033f9c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:27.457792 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05affe1b-95f9-4a70-8f76-01b12033f9c3 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:27.458454 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05affe1b-95f9-4a70-8f76-01b12033f9c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:27.459125 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05affe1b-95f9-4a70-8f76-01b12033f9c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:27.460124 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05affe1b-95f9-4a70-8f76-01b12033f9c3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:19:27.460684 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05affe1b-95f9-4a70-8f76-01b12033f9c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:27.472185 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05affe1b-95f9-4a70-8f76-01b12033f9c3 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:27.485788 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 120/227] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jun 7 14:19:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 429 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:27.496322 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05affe1b-95f9-4a70-8f76-01b12033f9c3 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:27.499202 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3a93c2-024a-4b70-9936-e37701e131fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:27.500325 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3a93c2-024a-4b70-9936-e37701e131fe None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:27.501003 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3a93c2-024a-4b70-9936-e37701e131fe None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:27.501493 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3a93c2-024a-4b70-9936-e37701e131fe None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:27.502884 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3a93c2-024a-4b70-9936-e37701e131fe None admin] RBAC: Adding query filter params (name=admin) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:19:27.503439 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3a93c2-024a-4b70-9936-e37701e131fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:27.506085 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3a93c2-024a-4b70-9936-e37701e131fe None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:27.526995 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 108/228] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jun 7 14:19:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 422 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:19:27.535616 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3a93c2-024a-4b70-9936-e37701e131fe None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:27.598954 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:27.635541 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:27.681724 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3a93c2-024a-4b70-9936-e37701e131fe None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:27.712157 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05affe1b-95f9-4a70-8f76-01b12033f9c3 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:27.752251 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a80fd35-3cef-45d0-bf3b-52fd4209a2a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:27.753178 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a80fd35-3cef-45d0-bf3b-52fd4209a2a0 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:27.753452 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a80fd35-3cef-45d0-bf3b-52fd4209a2a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:27.753720 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a80fd35-3cef-45d0-bf3b-52fd4209a2a0 None admin] PATH_INFO: `/v3/users/026b6f0bce8f4978919659f227dca61b` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:27.755156 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a80fd35-3cef-45d0-bf3b-52fd4209a2a0 None admin] RBAC: Authorizing `identity:get_user(user_id=026b6f0bce8f4978919659f227dca61b)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:27.756623 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a80fd35-3cef-45d0-bf3b-52fd4209a2a0 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:27.758881 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 109/229] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jun 7 14:19:27 2022] GET /identity/v3/users/026b6f0bce8f4978919659f227dca61b => generated 283 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:27.765002 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a80fd35-3cef-45d0-bf3b-52fd4209a2a0 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:27.791693 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a363d3bf-1773-4455-ac43-f4eb78d55583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:27.792624 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a363d3bf-1773-4455-ac43-f4eb78d55583 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:27.792912 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a363d3bf-1773-4455-ac43-f4eb78d55583 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:27.793197 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a363d3bf-1773-4455-ac43-f4eb78d55583 None admin] PATH_INFO: `/v3/users/026b6f0bce8f4978919659f227dca61b` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:27.795238 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a363d3bf-1773-4455-ac43-f4eb78d55583 None admin] RBAC: Authorizing `identity:get_user(user_id=026b6f0bce8f4978919659f227dca61b)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:27.796806 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a363d3bf-1773-4455-ac43-f4eb78d55583 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:27.799955 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 121/230] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jun 7 14:19:27 2022] GET /identity/v3/users/026b6f0bce8f4978919659f227dca61b => generated 283 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:19:27.808660 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a363d3bf-1773-4455-ac43-f4eb78d55583 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:27.972197 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a363d3bf-1773-4455-ac43-f4eb78d55583 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:27.995539 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:28.017170 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db14b9cd-03b8-4792-afcd-2dd8a49711fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:28.020974 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db14b9cd-03b8-4792-afcd-2dd8a49711fc None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:28.021239 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db14b9cd-03b8-4792-afcd-2dd8a49711fc None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:28.021532 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db14b9cd-03b8-4792-afcd-2dd8a49711fc None admin] PATH_INFO: `/v3/projects/f95de357ccad493984042656b2c0689a` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:28.024988 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db14b9cd-03b8-4792-afcd-2dd8a49711fc None admin] RBAC: Authorizing `identity:get_project(project_id=f95de357ccad493984042656b2c0689a)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:28.026581 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db14b9cd-03b8-4792-afcd-2dd8a49711fc None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:28.028693 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 122/231] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jun 7 14:19:27 2022] GET /identity/v3/projects/f95de357ccad493984042656b2c0689a => generated 302 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:28.045446 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db14b9cd-03b8-4792-afcd-2dd8a49711fc None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:28.056640 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a80fd35-3cef-45d0-bf3b-52fd4209a2a0 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:28.126591 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e16d28-a36b-4d22-8ea1-95b4c6a4b734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:28.127551 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e16d28-a36b-4d22-8ea1-95b4c6a4b734 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:28.127910 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e16d28-a36b-4d22-8ea1-95b4c6a4b734 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:28.128192 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e16d28-a36b-4d22-8ea1-95b4c6a4b734 None admin] PATH_INFO: `/v3/projects/f95de357ccad493984042656b2c0689a` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:28.132415 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e16d28-a36b-4d22-8ea1-95b4c6a4b734 None admin] RBAC: Authorizing `identity:get_project(project_id=f95de357ccad493984042656b2c0689a)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:28.135302 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e16d28-a36b-4d22-8ea1-95b4c6a4b734 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:28.137140 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 110/232] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jun 7 14:19:27 2022] GET /identity/v3/projects/f95de357ccad493984042656b2c0689a => generated 302 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:28.142975 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e16d28-a36b-4d22-8ea1-95b4c6a4b734 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:28.172822 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:28.333493 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db14b9cd-03b8-4792-afcd-2dd8a49711fc None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:28.387435 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bc08a6-2dcf-4155-be19-0c6f4b91f5db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:28.388679 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bc08a6-2dcf-4155-be19-0c6f4b91f5db None admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:28.388980 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bc08a6-2dcf-4155-be19-0c6f4b91f5db None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:28.389261 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bc08a6-2dcf-4155-be19-0c6f4b91f5db None admin] PATH_INFO: `/v3/projects/f95de357ccad493984042656b2c0689a/users/026b6f0bce8f4978919659f227dca61b/roles/494cd000d6884a179c334848f52a3a21` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:28.395651 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bc08a6-2dcf-4155-be19-0c6f4b91f5db None admin] RBAC: Authorizing `identity:create_grant(project_id=f95de357ccad493984042656b2c0689a, user_id=026b6f0bce8f4978919659f227dca61b, role_id=494cd000d6884a179c334848f52a3a21)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:28.397147 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bc08a6-2dcf-4155-be19-0c6f4b91f5db None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:28.424654 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e16d28-a36b-4d22-8ea1-95b4c6a4b734 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:28.463857 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5075499-c14d-40c9-9769-e8e2e313c43b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:28.464851 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5075499-c14d-40c9-9769-e8e2e313c43b None admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:28.465171 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5075499-c14d-40c9-9769-e8e2e313c43b None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:28.465638 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5075499-c14d-40c9-9769-e8e2e313c43b None admin] PATH_INFO: `/v3/projects/f95de357ccad493984042656b2c0689a/users/026b6f0bce8f4978919659f227dca61b/roles/9baff2b18f2649f7b757ff2e1f63c853` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:28.468559 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5075499-c14d-40c9-9769-e8e2e313c43b None admin] RBAC: Authorizing `identity:create_grant(project_id=f95de357ccad493984042656b2c0689a, user_id=026b6f0bce8f4978919659f227dca61b, role_id=9baff2b18f2649f7b757ff2e1f63c853)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:28.470508 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5075499-c14d-40c9-9769-e8e2e313c43b None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:28.495631 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 123/233] 10.222.0.85 () {58 vars in 1439 bytes} [Tue Jun 7 14:19:28 2022] PUT /identity/v3/projects/f95de357ccad493984042656b2c0689a/users/026b6f0bce8f4978919659f227dca61b/roles/494cd000d6884a179c334848f52a3a21 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:19:28.569261 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 111/234] 10.222.0.85 () {58 vars in 1439 bytes} [Tue Jun 7 14:19:28 2022] PUT /identity/v3/projects/f95de357ccad493984042656b2c0689a/users/026b6f0bce8f4978919659f227dca61b/roles/9baff2b18f2649f7b757ff2e1f63c853 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:19:29.710592 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab2a350-5225-486e-bf38-4d7772deca75 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:29.711530 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab2a350-5225-486e-bf38-4d7772deca75 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:29.712025 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab2a350-5225-486e-bf38-4d7772deca75 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:29.714158 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 124/235] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:19:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:19:29.720326 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be6896d-47b6-451d-986c-cfed1b77682c None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:29.721483 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be6896d-47b6-451d-986c-cfed1b77682c None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:29.721986 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be6896d-47b6-451d-986c-cfed1b77682c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:29.741001 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-0be6896d-47b6-451d-986c-cfed1b77682c None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:19:29.742622 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-0be6896d-47b6-451d-986c-cfed1b77682c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:19:29.814890 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:29.825962 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 112/236] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:19:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 14:19:29.842699 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769dc175-097a-4bb2-8023-b1cff36e8583 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:29.843155 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769dc175-097a-4bb2-8023-b1cff36e8583 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:29.843640 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769dc175-097a-4bb2-8023-b1cff36e8583 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:29.845166 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 125/237] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:19:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 14:19:29.856608 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c6146-c396-4fe0-b0d2-cc2c3412b36a None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:29.856979 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c6146-c396-4fe0-b0d2-cc2c3412b36a None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:29.857322 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c6146-c396-4fe0-b0d2-cc2c3412b36a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:29.883720 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-5e6c6146-c396-4fe0-b0d2-cc2c3412b36a None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:19:29.885247 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769dc175-097a-4bb2-8023-b1cff36e8583 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:29.886273 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-5e6c6146-c396-4fe0-b0d2-cc2c3412b36a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:19:29.976428 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 113/238] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:19:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:19:30.005155 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6c6146-c396-4fe0-b0d2-cc2c3412b36a None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:30.064836 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:30.191673 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6c6146-c396-4fe0-b0d2-cc2c3412b36a None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:30.257032 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769dc175-097a-4bb2-8023-b1cff36e8583 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:30.258922 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86f370c-b64c-4ec2-8b80-ab21c30c69fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:30.260869 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86f370c-b64c-4ec2-8b80-ab21c30c69fa None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:30.261589 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86f370c-b64c-4ec2-8b80-ab21c30c69fa None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:30.262288 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86f370c-b64c-4ec2-8b80-ab21c30c69fa None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:30.268657 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:30.289555 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86f370c-b64c-4ec2-8b80-ab21c30c69fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:19:30.306729 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-a86f370c-b64c-4ec2-8b80-ab21c30c69fa None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 14:19:30.306729 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:19:30.306729 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:19:30.306729 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:19:30.306729 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:19:30.306729 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:19:30.306729 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:19:30.306729 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:19:30.306729 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:19:30.306729 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:19:30.306729 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:19:30.306729 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:19:30.306729 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:19:30.306729 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:19:30.306729 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:19:30.306729 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:19:30.307692 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:19:30.307692 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:19:30.307692 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:19:30.307692 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:19:30.307692 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:19:30.307692 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:19:30.307692 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:19:30.307692 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:19:30.307692 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:19:30.307692 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:19:30.307692 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:19:30.307692 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:19:30.307692 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:19:30.307692 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:19:30.307692 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:19:30.307692 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:19:30.308510 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:19:30.308510 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:19:30.308510 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:19:30.308510 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:19:30.308510 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:19:30.308510 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:19:30.308510 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:19:30.308510 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:19:30.308510 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:19:30.308510 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:19:30.308510 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 14:19:30.308510 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:19:30.309408 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 114/239] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jun 7 14:19:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:19:30.311941 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3116bf-9cfb-42ac-bb7c-cde95446e408 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:30.312884 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3116bf-9cfb-42ac-bb7c-cde95446e408 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:30.313664 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3116bf-9cfb-42ac-bb7c-cde95446e408 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:30.316669 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 115/240] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:19:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:19:30.318290 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3116bf-9cfb-42ac-bb7c-cde95446e408 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:30.346707 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec280f7-5b90-421d-9909-3b38d1eea477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:30.347571 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec280f7-5b90-421d-9909-3b38d1eea477 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:30.347854 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec280f7-5b90-421d-9909-3b38d1eea477 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:30.348129 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec280f7-5b90-421d-9909-3b38d1eea477 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:30.349366 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec280f7-5b90-421d-9909-3b38d1eea477 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:30.350813 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec280f7-5b90-421d-9909-3b38d1eea477 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:30.353352 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 126/241] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:19:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 469 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:30.354547 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b88902-168a-4a5b-a50f-17adba5aaeae None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:30.354825 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b88902-168a-4a5b-a50f-17adba5aaeae None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:30.355266 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b88902-168a-4a5b-a50f-17adba5aaeae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:30.377013 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-a1b88902-168a-4a5b-a50f-17adba5aaeae None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:19:30.380117 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-a1b88902-168a-4a5b-a50f-17adba5aaeae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:19:30.476126 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 127/242] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:19:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:19:30.478880 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b88902-168a-4a5b-a50f-17adba5aaeae None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:30.481892 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3116bf-9cfb-42ac-bb7c-cde95446e408 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:30.530708 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef400986-82e8-41cf-959e-8f75ad43bb5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:30.531854 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef400986-82e8-41cf-959e-8f75ad43bb5a None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:30.532358 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef400986-82e8-41cf-959e-8f75ad43bb5a None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:30.533095 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef400986-82e8-41cf-959e-8f75ad43bb5a None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:30.534168 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef400986-82e8-41cf-959e-8f75ad43bb5a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:19:30.534733 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef400986-82e8-41cf-959e-8f75ad43bb5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:30.536610 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef400986-82e8-41cf-959e-8f75ad43bb5a None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:30.544067 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:30.547602 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 116/243] 10.222.0.85 () {56 vars in 1101 bytes} [Tue Jun 7 14:19:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:30.548948 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef400986-82e8-41cf-959e-8f75ad43bb5a None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:30.619804 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b88902-168a-4a5b-a50f-17adba5aaeae None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:30.633544 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:30.672404 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ec0d8d-9eb9-430d-8562-a0befc8dac6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:30.672404 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ec0d8d-9eb9-430d-8562-a0befc8dac6c None admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:30.672404 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ec0d8d-9eb9-430d-8562-a0befc8dac6c None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:30.672404 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ec0d8d-9eb9-430d-8562-a0befc8dac6c None admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:30.673079 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ec0d8d-9eb9-430d-8562-a0befc8dac6c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:30.673079 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ec0d8d-9eb9-430d-8562-a0befc8dac6c None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:30.707800 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef400986-82e8-41cf-959e-8f75ad43bb5a None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:30.753752 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 128/244] 10.222.0.85 () {60 vars in 1132 bytes} [Tue Jun 7 14:19:30 2022] POST /identity/v3/users => generated 293 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:19:30.755440 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ec0d8d-9eb9-430d-8562-a0befc8dac6c None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:30.767784 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b6a4b5-bf5b-42e0-b06c-dedf557de95f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:30.768831 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b6a4b5-bf5b-42e0-b06c-dedf557de95f None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:30.769123 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b6a4b5-bf5b-42e0-b06c-dedf557de95f None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:30.769459 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b6a4b5-bf5b-42e0-b06c-dedf557de95f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:30.785029 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b6a4b5-bf5b-42e0-b06c-dedf557de95f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:19:30.795457 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-33b6a4b5-bf5b-42e0-b06c-dedf557de95f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 14:19:30.795457 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:19:30.795457 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:19:30.795457 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:19:30.795457 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:19:30.795457 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:19:30.795457 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:19:30.795457 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:19:30.795457 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:19:30.795457 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:19:30.795457 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:19:30.795457 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:19:30.795457 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:19:30.795457 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:19:30.795457 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:19:30.795457 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:19:30.796448 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:19:30.796448 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:19:30.796448 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:19:30.796448 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:19:30.796448 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:19:30.796448 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:19:30.796448 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:19:30.796448 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:19:30.796448 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:19:30.796448 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:19:30.796448 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:19:30.796448 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:19:30.796448 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:19:30.796448 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:19:30.796448 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:19:30.796448 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:19:30.797232 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:19:30.797232 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:19:30.797232 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:19:30.797232 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:19:30.797232 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:19:30.797232 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:19:30.797232 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:19:30.797232 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:19:30.797232 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:19:30.797232 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:19:30.797232 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 14:19:30.797232 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:19:30.797232 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 117/245] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jun 7 14:19:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:19:30.808113 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b6a4b5-bf5b-42e0-b06c-dedf557de95f None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:30.958223 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ec0d8d-9eb9-430d-8562-a0befc8dac6c None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:30.971499 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:31.032739 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b6a4b5-bf5b-42e0-b06c-dedf557de95f None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:31.046484 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62508982-6f7c-4bc5-a2a4-57a917c31992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:31.049173 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:31.053874 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62508982-6f7c-4bc5-a2a4-57a917c31992 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:31.054361 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62508982-6f7c-4bc5-a2a4-57a917c31992 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:31.054845 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62508982-6f7c-4bc5-a2a4-57a917c31992 None admin] PATH_INFO: `/v3/users/aa668fea4a7c464eac1ba149b34b69b6` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:31.089619 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ad241f-a94f-407f-afce-f30faa9f2f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:31.090322 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ad241f-a94f-407f-afce-f30faa9f2f72 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:31.090609 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ad241f-a94f-407f-afce-f30faa9f2f72 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:31.090895 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ad241f-a94f-407f-afce-f30faa9f2f72 None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:31.091667 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ad241f-a94f-407f-afce-f30faa9f2f72 None admin] RBAC: Adding query filter params (name=member) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:19:31.091978 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ad241f-a94f-407f-afce-f30faa9f2f72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:31.093554 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62508982-6f7c-4bc5-a2a4-57a917c31992 None admin] RBAC: Authorizing `identity:get_user(user_id=aa668fea4a7c464eac1ba149b34b69b6)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:31.097816 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62508982-6f7c-4bc5-a2a4-57a917c31992 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:31.100471 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 129/246] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jun 7 14:19:30 2022] GET /identity/v3/users/aa668fea4a7c464eac1ba149b34b69b6 => generated 297 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:31.101679 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ad241f-a94f-407f-afce-f30faa9f2f72 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:31.110491 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62508982-6f7c-4bc5-a2a4-57a917c31992 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:31.123349 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 118/247] 10.222.0.85 () {56 vars in 1101 bytes} [Tue Jun 7 14:19:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:31.129996 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ad241f-a94f-407f-afce-f30faa9f2f72 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:31.257681 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62508982-6f7c-4bc5-a2a4-57a917c31992 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:31.300181 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ad241f-a94f-407f-afce-f30faa9f2f72 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:31.316985 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dac203-4c13-4672-a4f5-f0215192dcc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:31.318071 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dac203-4c13-4672-a4f5-f0215192dcc9 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:31.318606 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dac203-4c13-4672-a4f5-f0215192dcc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:31.319091 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dac203-4c13-4672-a4f5-f0215192dcc9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:31.362860 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:31.366670 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce548c8-5183-497c-bcc8-f5117a3ed607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:31.367875 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce548c8-5183-497c-bcc8-f5117a3ed607 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:31.368314 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce548c8-5183-497c-bcc8-f5117a3ed607 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:31.368824 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce548c8-5183-497c-bcc8-f5117a3ed607 None admin] PATH_INFO: `/v3/users/5a235d1390ea42c9b56f16bffef3d4e0` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:31.372566 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce548c8-5183-497c-bcc8-f5117a3ed607 None admin] RBAC: Authorizing `identity:get_user(user_id=5a235d1390ea42c9b56f16bffef3d4e0)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:31.376142 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce548c8-5183-497c-bcc8-f5117a3ed607 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:31.379991 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 119/248] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jun 7 14:19:31 2022] GET /identity/v3/users/5a235d1390ea42c9b56f16bffef3d4e0 => generated 297 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:19:31.389498 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce548c8-5183-497c-bcc8-f5117a3ed607 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:31.442397 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dac203-4c13-4672-a4f5-f0215192dcc9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:31.444181 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dac203-4c13-4672-a4f5-f0215192dcc9 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:31.451893 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:31.535142 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce548c8-5183-497c-bcc8-f5117a3ed607 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:31.576554 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59673f3-34ac-46dc-a8d5-798fc698ba11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:31.577992 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59673f3-34ac-46dc-a8d5-798fc698ba11 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:31.578563 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 130/249] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jun 7 14:19:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 07 14:19:31.578701 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59673f3-34ac-46dc-a8d5-798fc698ba11 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:31.583331 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59673f3-34ac-46dc-a8d5-798fc698ba11 None admin] PATH_INFO: `/v3/projects/f95de357ccad493984042656b2c0689a` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:31.583331 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59673f3-34ac-46dc-a8d5-798fc698ba11 None admin] RBAC: Authorizing `identity:get_project(project_id=f95de357ccad493984042656b2c0689a)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:31.583495 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59673f3-34ac-46dc-a8d5-798fc698ba11 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:31.585113 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dac203-4c13-4672-a4f5-f0215192dcc9 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:31.586138 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 120/250] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jun 7 14:19:31 2022] GET /identity/v3/projects/f95de357ccad493984042656b2c0689a => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:19:31.599153 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59673f3-34ac-46dc-a8d5-798fc698ba11 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:31.676774 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:31.743994 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59673f3-34ac-46dc-a8d5-798fc698ba11 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:31.759808 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:31.797974 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc826a5c-91f7-4efe-9000-16e47467b4df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:31.799090 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc826a5c-91f7-4efe-9000-16e47467b4df None admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:31.799292 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc826a5c-91f7-4efe-9000-16e47467b4df None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:31.799842 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc826a5c-91f7-4efe-9000-16e47467b4df None admin] PATH_INFO: `/v3/projects/f95de357ccad493984042656b2c0689a/users/5a235d1390ea42c9b56f16bffef3d4e0/roles/3771341b3be641139e752aab690397a4` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:31.803939 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc826a5c-91f7-4efe-9000-16e47467b4df None admin] RBAC: Authorizing `identity:create_grant(project_id=f95de357ccad493984042656b2c0689a, user_id=5a235d1390ea42c9b56f16bffef3d4e0, role_id=3771341b3be641139e752aab690397a4)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:31.806034 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc826a5c-91f7-4efe-9000-16e47467b4df None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:31.827057 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dac203-4c13-4672-a4f5-f0215192dcc9 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:31.875133 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 121/251] 10.222.0.85 () {58 vars in 1439 bytes} [Tue Jun 7 14:19:31 2022] PUT /identity/v3/projects/f95de357ccad493984042656b2c0689a/users/5a235d1390ea42c9b56f16bffef3d4e0/roles/3771341b3be641139e752aab690397a4 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:19:31.897977 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a828bd-4fb5-4524-9505-2daa82df7280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:31.899293 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a828bd-4fb5-4524-9505-2daa82df7280 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:31.899804 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a828bd-4fb5-4524-9505-2daa82df7280 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:31.900302 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a828bd-4fb5-4524-9505-2daa82df7280 None admin] PATH_INFO: `/v3/projects/f95de357ccad493984042656b2c0689a` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:31.905434 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a828bd-4fb5-4524-9505-2daa82df7280 None admin] RBAC: Authorizing `identity:get_project(project_id=f95de357ccad493984042656b2c0689a)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:31.907239 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a828bd-4fb5-4524-9505-2daa82df7280 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:31.909825 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 131/252] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jun 7 14:19:31 2022] GET /identity/v3/projects/f95de357ccad493984042656b2c0689a => generated 302 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:19:31.916299 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc826a5c-91f7-4efe-9000-16e47467b4df None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:32.056412 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a20352-2f5e-40e1-9557-7f77106f5190 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:32.056701 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a20352-2f5e-40e1-9557-7f77106f5190 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:32.056987 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a20352-2f5e-40e1-9557-7f77106f5190 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:32.058579 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 132/253] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:19:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:19:32.064626 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a4f2ed-edcf-47c4-b138-bce836562501 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:32.064921 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a4f2ed-edcf-47c4-b138-bce836562501 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:32.065265 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a4f2ed-edcf-47c4-b138-bce836562501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:32.103620 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-31a4f2ed-edcf-47c4-b138-bce836562501 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:19:32.104900 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-31a4f2ed-edcf-47c4-b138-bce836562501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:19:32.110888 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc826a5c-91f7-4efe-9000-16e47467b4df None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:32.209415 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c6f43c-d324-4158-a3ab-38071d990bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:32.210108 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6f43c-d324-4158-a3ab-38071d990bee None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:32.210401 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6f43c-d324-4158-a3ab-38071d990bee None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:32.210774 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6f43c-d324-4158-a3ab-38071d990bee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:32.211737 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c6f43c-d324-4158-a3ab-38071d990bee None admin] RBAC: Adding query filter params (role.id=e2072a2a35d64eec9d32dac7c47c4419, scope.project.id=f95de357ccad493984042656b2c0689a, user.id=aa668fea4a7c464eac1ba149b34b69b6) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:19:32.212059 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c6f43c-d324-4158-a3ab-38071d990bee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:32.213686 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c6f43c-d324-4158-a3ab-38071d990bee None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:32.226678 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 122/254] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jun 7 14:19:31 2022] GET /identity/v3/role_assignments?user.id=aa668fea4a7c464eac1ba149b34b69b6&scope.project.id=f95de357ccad493984042656b2c0689a&role.id=e2072a2a35d64eec9d32dac7c47c4419 => generated 259 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:32.256078 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 133/255] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:19:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:19:32.289280 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c6f43c-d324-4158-a3ab-38071d990bee None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:32.351733 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:32.454018 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c6f43c-d324-4158-a3ab-38071d990bee None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:32.511121 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba2f3c9-8c77-4c1b-8d3d-a5452223fa61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:32.511945 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba2f3c9-8c77-4c1b-8d3d-a5452223fa61 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:32.512227 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba2f3c9-8c77-4c1b-8d3d-a5452223fa61 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:32.512535 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba2f3c9-8c77-4c1b-8d3d-a5452223fa61 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:32.531829 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba2f3c9-8c77-4c1b-8d3d-a5452223fa61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:19:32.540285 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-7ba2f3c9-8c77-4c1b-8d3d-a5452223fa61 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:19:32.540285 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:19:32.540285 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:19:32.540285 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:19:32.540285 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:19:32.540285 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:19:32.540285 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:19:32.540285 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:19:32.540285 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:19:32.540285 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:19:32.540285 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:19:32.540285 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:19:32.540285 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:19:32.540285 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:19:32.540285 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:19:32.540285 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:19:32.542088 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:19:32.542088 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:19:32.542088 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:19:32.542088 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:19:32.542088 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:19:32.542088 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:19:32.542088 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:19:32.542088 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:19:32.542088 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:19:32.542088 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:19:32.542088 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:19:32.542088 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:19:32.542088 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:19:32.542088 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:19:32.542088 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:19:32.542088 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:19:32.544308 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:19:32.544308 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:19:32.544308 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:19:32.544308 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:19:32.544308 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:19:32.544308 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:19:32.544308 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:19:32.544308 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:19:32.544308 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:19:32.544308 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:19:32.544308 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:19:32.544308 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:19:32.544308 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 123/256] 10.222.0.85 () {56 vars in 1096 bytes} [Tue Jun 7 14:19:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:19:32.553960 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a4f2ed-edcf-47c4-b138-bce836562501 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:32.562861 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf712e0a-5abf-4359-ab2e-a69ba7d3e088 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:32.563316 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf712e0a-5abf-4359-ab2e-a69ba7d3e088 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:32.563465 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf712e0a-5abf-4359-ab2e-a69ba7d3e088 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:32.565128 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 124/257] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:19:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 14:19:32.583249 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef507d7-4289-4481-aee6-510442a1f414 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:32.583699 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef507d7-4289-4481-aee6-510442a1f414 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:32.584435 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef507d7-4289-4481-aee6-510442a1f414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:32.625619 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-aef507d7-4289-4481-aee6-510442a1f414 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:19:32.626896 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-aef507d7-4289-4481-aee6-510442a1f414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:19:32.640657 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:32.703050 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a4f2ed-edcf-47c4-b138-bce836562501 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:32.748799 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0642424d-38ba-454c-a668-2bab8e95b7d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:32.749399 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0642424d-38ba-454c-a668-2bab8e95b7d9 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:32.749687 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0642424d-38ba-454c-a668-2bab8e95b7d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:32.750923 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0642424d-38ba-454c-a668-2bab8e95b7d9 None admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:32.751716 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0642424d-38ba-454c-a668-2bab8e95b7d9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:19:32.752029 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0642424d-38ba-454c-a668-2bab8e95b7d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:32.753475 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0642424d-38ba-454c-a668-2bab8e95b7d9 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:32.764221 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 134/258] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jun 7 14:19:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:32.775793 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0642424d-38ba-454c-a668-2bab8e95b7d9 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:32.890785 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 125/259] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:19:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 14:19:32.934159 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef507d7-4289-4481-aee6-510442a1f414 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:32.964952 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0642424d-38ba-454c-a668-2bab8e95b7d9 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:33.002286 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:33.034178 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eabec3-d762-4d0b-90c8-ad4266c302b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:33.035097 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eabec3-d762-4d0b-90c8-ad4266c302b9 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:33.035536 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:33.036654 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eabec3-d762-4d0b-90c8-ad4266c302b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:33.036993 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eabec3-d762-4d0b-90c8-ad4266c302b9 None admin] PATH_INFO: `/v3/users/026b6f0bce8f4978919659f227dca61b` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:33.038303 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63eabec3-d762-4d0b-90c8-ad4266c302b9 None admin] RBAC: Authorizing `identity:get_user(user_id=026b6f0bce8f4978919659f227dca61b)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:33.040181 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63eabec3-d762-4d0b-90c8-ad4266c302b9 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:33.042469 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 135/260] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jun 7 14:19:32 2022] GET /identity/v3/users/026b6f0bce8f4978919659f227dca61b => generated 283 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:33.049703 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eabec3-d762-4d0b-90c8-ad4266c302b9 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:33.271919 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eabec3-d762-4d0b-90c8-ad4266c302b9 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:33.294655 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef507d7-4289-4481-aee6-510442a1f414 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:33.360217 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f045405-72bf-406b-b345-a33d51b05678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:33.361067 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f045405-72bf-406b-b345-a33d51b05678 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:33.361362 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f045405-72bf-406b-b345-a33d51b05678 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:33.361697 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f045405-72bf-406b-b345-a33d51b05678 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:33.369356 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbf78d2-cbe4-452d-8d48-1ce7f8a832e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:33.370293 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbf78d2-cbe4-452d-8d48-1ce7f8a832e6 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:33.370623 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbf78d2-cbe4-452d-8d48-1ce7f8a832e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:33.370901 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbf78d2-cbe4-452d-8d48-1ce7f8a832e6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:33.403325 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbf78d2-cbe4-452d-8d48-1ce7f8a832e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:19:33.411713 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-8fbf78d2-cbe4-452d-8d48-1ce7f8a832e6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 14:19:33.411713 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:19:33.411713 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:19:33.411713 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:19:33.411713 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:19:33.411713 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:19:33.411713 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:19:33.411713 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:19:33.411713 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:19:33.411713 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:19:33.411713 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:19:33.411713 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:19:33.411713 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:19:33.411713 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:19:33.411713 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:19:33.411713 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:19:33.412729 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:19:33.412729 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:19:33.412729 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:19:33.412729 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:19:33.412729 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:19:33.412729 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:19:33.412729 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:19:33.412729 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:19:33.412729 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:19:33.412729 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:19:33.412729 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:19:33.412729 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:19:33.412729 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:19:33.412729 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:19:33.412729 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:19:33.412729 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:19:33.413522 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:19:33.413522 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:19:33.413522 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:19:33.413522 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:19:33.413522 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:19:33.413522 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:19:33.413522 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:19:33.413522 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:19:33.413522 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:19:33.413522 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:19:33.413522 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 14:19:33.413522 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:19:33.428826 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 126/261] 10.222.0.85 () {56 vars in 1114 bytes} [Tue Jun 7 14:19:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 496 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:19:33.433860 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbf78d2-cbe4-452d-8d48-1ce7f8a832e6 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:33.549137 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:33.585556 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f045405-72bf-406b-b345-a33d51b05678 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:33.587168 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f045405-72bf-406b-b345-a33d51b05678 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:33.662299 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:33.700820 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbf78d2-cbe4-452d-8d48-1ce7f8a832e6 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:33.769177 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9163a6c4-c6c6-4c47-9829-90c8f321a3f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:33.770026 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9163a6c4-c6c6-4c47-9829-90c8f321a3f2 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:33.770318 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9163a6c4-c6c6-4c47-9829-90c8f321a3f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:33.770596 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9163a6c4-c6c6-4c47-9829-90c8f321a3f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:33.771356 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9163a6c4-c6c6-4c47-9829-90c8f321a3f2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:19:33.771672 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9163a6c4-c6c6-4c47-9829-90c8f321a3f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:33.773273 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9163a6c4-c6c6-4c47-9829-90c8f321a3f2 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:33.792907 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 127/262] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jun 7 14:19:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:33.798966 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9163a6c4-c6c6-4c47-9829-90c8f321a3f2 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:33.863423 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 136/263] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jun 7 14:19:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 815 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 14:19:33.872045 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f045405-72bf-406b-b345-a33d51b05678 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:34.018420 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9163a6c4-c6c6-4c47-9829-90c8f321a3f2 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:34.096955 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49622f7-fed1-4803-85c0-5798710b7073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:34.098991 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49622f7-fed1-4803-85c0-5798710b7073 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:34.099777 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49622f7-fed1-4803-85c0-5798710b7073 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:34.100590 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49622f7-fed1-4803-85c0-5798710b7073 None admin] PATH_INFO: `/v3/users/026b6f0bce8f4978919659f227dca61b` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:34.106259 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49622f7-fed1-4803-85c0-5798710b7073 None admin] RBAC: Authorizing `identity:get_user(user_id=026b6f0bce8f4978919659f227dca61b)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:34.109377 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49622f7-fed1-4803-85c0-5798710b7073 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:34.112533 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 128/264] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jun 7 14:19:33 2022] GET /identity/v3/users/026b6f0bce8f4978919659f227dca61b => generated 283 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:19:34.118592 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49622f7-fed1-4803-85c0-5798710b7073 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:34.147450 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:34.156777 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:34.198821 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f045405-72bf-406b-b345-a33d51b05678 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:34.297969 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407dce25-9e95-429c-b55b-8ef621644593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:34.299744 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407dce25-9e95-429c-b55b-8ef621644593 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:34.300195 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407dce25-9e95-429c-b55b-8ef621644593 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:34.300672 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407dce25-9e95-429c-b55b-8ef621644593 None admin] PATH_INFO: `/v3/projects/f95de357ccad493984042656b2c0689a` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:34.305816 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407dce25-9e95-429c-b55b-8ef621644593 None admin] RBAC: Authorizing `identity:get_project(project_id=f95de357ccad493984042656b2c0689a)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:34.307609 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407dce25-9e95-429c-b55b-8ef621644593 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:34.312074 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 137/265] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jun 7 14:19:33 2022] GET /identity/v3/projects/f95de357ccad493984042656b2c0689a => generated 302 bytes in 439 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:34.320447 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407dce25-9e95-429c-b55b-8ef621644593 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:34.349884 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49622f7-fed1-4803-85c0-5798710b7073 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:34.429376 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfe0497-5810-4b35-9e60-52ef95dae9a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:34.431060 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfe0497-5810-4b35-9e60-52ef95dae9a5 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:34.431556 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfe0497-5810-4b35-9e60-52ef95dae9a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:34.432039 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfe0497-5810-4b35-9e60-52ef95dae9a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:34.545141 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407dce25-9e95-429c-b55b-8ef621644593 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:34.609215 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d08318-4671-43eb-b570-520e73d23a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:34.610601 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d08318-4671-43eb-b570-520e73d23a3f None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:34.611291 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d08318-4671-43eb-b570-520e73d23a3f None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:34.611943 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d08318-4671-43eb-b570-520e73d23a3f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:34.613767 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d08318-4671-43eb-b570-520e73d23a3f None admin] RBAC: Adding query filter params (role.id=9baff2b18f2649f7b757ff2e1f63c853, scope.project.id=f95de357ccad493984042656b2c0689a, user.id=026b6f0bce8f4978919659f227dca61b) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:19:34.614559 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d08318-4671-43eb-b570-520e73d23a3f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:34.618197 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d08318-4671-43eb-b570-520e73d23a3f None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:34.628837 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfe0497-5810-4b35-9e60-52ef95dae9a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:34.630717 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 138/266] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jun 7 14:19:34 2022] GET /identity/v3/role_assignments?user.id=026b6f0bce8f4978919659f227dca61b&scope.project.id=f95de357ccad493984042656b2c0689a&role.id=9baff2b18f2649f7b757ff2e1f63c853 => generated 608 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:34.631892 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfe0497-5810-4b35-9e60-52ef95dae9a5 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:34.671506 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:34.841086 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 129/267] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jun 7 14:19:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 723 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 07 14:19:34.849412 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d08318-4671-43eb-b570-520e73d23a3f None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:34.892763 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:35.053349 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d08318-4671-43eb-b570-520e73d23a3f None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:35.097116 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d58ca8-3567-48a4-a850-ca09e0689a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:35.098589 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d58ca8-3567-48a4-a850-ca09e0689a8f None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:35.099287 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d58ca8-3567-48a4-a850-ca09e0689a8f None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:35.099826 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d58ca8-3567-48a4-a850-ca09e0689a8f None admin] PATH_INFO: `/v3/projects/f95de357ccad493984042656b2c0689a` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:35.101604 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d58ca8-3567-48a4-a850-ca09e0689a8f None admin] RBAC: Authorizing `identity:get_project(project_id=f95de357ccad493984042656b2c0689a)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:35.103381 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d58ca8-3567-48a4-a850-ca09e0689a8f None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:35.105496 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 139/268] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jun 7 14:19:34 2022] GET /identity/v3/projects/f95de357ccad493984042656b2c0689a => generated 302 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:35.115150 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfe0497-5810-4b35-9e60-52ef95dae9a5 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:35.234651 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cb7dba-f5f3-4159-9d75-557b6f80699f None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:35.235512 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cb7dba-f5f3-4159-9d75-557b6f80699f None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:35.235966 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cb7dba-f5f3-4159-9d75-557b6f80699f None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:35.237739 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 140/269] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:19:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 14:19:35.247700 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29254fe8-d2ae-41a7-9097-93729eefcb89 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:35.248330 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29254fe8-d2ae-41a7-9097-93729eefcb89 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:35.248812 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29254fe8-d2ae-41a7-9097-93729eefcb89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:35.300791 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-29254fe8-d2ae-41a7-9097-93729eefcb89 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:19:35.302974 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-29254fe8-d2ae-41a7-9097-93729eefcb89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:19:35.330994 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfe0497-5810-4b35-9e60-52ef95dae9a5 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:35.371412 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:35.422300 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec4416b-7461-4503-b2b9-90c33b9b1bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:35.423306 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec4416b-7461-4503-b2b9-90c33b9b1bd5 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:35.423826 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec4416b-7461-4503-b2b9-90c33b9b1bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:35.424319 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec4416b-7461-4503-b2b9-90c33b9b1bd5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:35.425537 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec4416b-7461-4503-b2b9-90c33b9b1bd5 None admin] RBAC: Adding query filter params (role.id=494cd000d6884a179c334848f52a3a21, scope.project.id=f95de357ccad493984042656b2c0689a, user.id=026b6f0bce8f4978919659f227dca61b) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:19:35.426047 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec4416b-7461-4503-b2b9-90c33b9b1bd5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:35.427709 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec4416b-7461-4503-b2b9-90c33b9b1bd5 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:35.440842 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 130/270] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jun 7 14:19:35 2022] GET /identity/v3/role_assignments?user.id=026b6f0bce8f4978919659f227dca61b&scope.project.id=f95de357ccad493984042656b2c0689a&role.id=494cd000d6884a179c334848f52a3a21 => generated 608 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:35.442207 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 141/271] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:19:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:19:35.470147 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec4416b-7461-4503-b2b9-90c33b9b1bd5 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:35.643505 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec4416b-7461-4503-b2b9-90c33b9b1bd5 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:35.693714 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108a8431-8863-48b3-b4c0-af430eeb6d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:35.694792 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108a8431-8863-48b3-b4c0-af430eeb6d66 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:35.695299 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108a8431-8863-48b3-b4c0-af430eeb6d66 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:35.696363 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108a8431-8863-48b3-b4c0-af430eeb6d66 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:35.718737 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108a8431-8863-48b3-b4c0-af430eeb6d66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:19:35.727783 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-108a8431-8863-48b3-b4c0-af430eeb6d66 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 14:19:35.727783 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:19:35.727783 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:19:35.727783 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:19:35.727783 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:19:35.727783 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:19:35.727783 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:19:35.727783 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:19:35.727783 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:19:35.727783 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:19:35.727783 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:19:35.727783 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:19:35.727783 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:19:35.727783 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:19:35.727783 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:19:35.727783 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:19:35.728960 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:19:35.728960 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:19:35.728960 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:19:35.728960 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:19:35.728960 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:19:35.728960 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:19:35.728960 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:19:35.728960 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:19:35.728960 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:19:35.728960 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:19:35.728960 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:19:35.728960 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:19:35.728960 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:19:35.728960 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:19:35.728960 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:19:35.728960 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:19:35.729981 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:19:35.729981 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:19:35.729981 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:19:35.729981 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:19:35.729981 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:19:35.729981 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:19:35.729981 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:19:35.729981 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:19:35.729981 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:19:35.729981 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:19:35.729981 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 14:19:35.729981 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:19:35.738133 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 131/272] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jun 7 14:19:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:19:35.746228 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29254fe8-d2ae-41a7-9097-93729eefcb89 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:35.758192 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ad9b27-7ef1-4a53-a609-036e3f0bff8d None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:35.759636 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ad9b27-7ef1-4a53-a609-036e3f0bff8d None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:35.760151 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ad9b27-7ef1-4a53-a609-036e3f0bff8d None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:35.761919 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 132/273] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:19:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:19:35.771339 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f130bd-435f-423d-ba16-078acbccfba8 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:35.771973 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f130bd-435f-423d-ba16-078acbccfba8 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:35.772530 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f130bd-435f-423d-ba16-078acbccfba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:35.778766 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:35.794792 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-23f130bd-435f-423d-ba16-078acbccfba8 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:19:35.796328 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-23f130bd-435f-423d-ba16-078acbccfba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:19:35.854315 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:35.906470 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29254fe8-d2ae-41a7-9097-93729eefcb89 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:35.920591 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 133/274] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:19:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 14:19:35.961274 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f130bd-435f-423d-ba16-078acbccfba8 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:35.962305 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5d173c-e732-4f15-9d79-1f37d49c5a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:35.968409 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5d173c-e732-4f15-9d79-1f37d49c5a94 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:35.968409 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5d173c-e732-4f15-9d79-1f37d49c5a94 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:35.968409 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5d173c-e732-4f15-9d79-1f37d49c5a94 None admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:35.968409 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5d173c-e732-4f15-9d79-1f37d49c5a94 None admin] RBAC: Adding query filter params (name=member) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:19:35.968409 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5d173c-e732-4f15-9d79-1f37d49c5a94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:35.968409 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5d173c-e732-4f15-9d79-1f37d49c5a94 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:35.983763 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 142/275] 10.222.0.85 () {56 vars in 1101 bytes} [Tue Jun 7 14:19:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:19:35.994866 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5d173c-e732-4f15-9d79-1f37d49c5a94 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:36.085124 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f130bd-435f-423d-ba16-078acbccfba8 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:36.118213 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:36.137414 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3623e353-510a-4e5e-95d9-1a91de0bb0df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:36.139295 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3623e353-510a-4e5e-95d9-1a91de0bb0df None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:36.139818 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3623e353-510a-4e5e-95d9-1a91de0bb0df None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:36.140406 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3623e353-510a-4e5e-95d9-1a91de0bb0df None admin] PATH_INFO: `/v3/domains/default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:36.142094 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3623e353-510a-4e5e-95d9-1a91de0bb0df None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:36.143881 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5d173c-e732-4f15-9d79-1f37d49c5a94 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:36.144325 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3623e353-510a-4e5e-95d9-1a91de0bb0df None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:36.147550 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 134/276] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:19:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:36.156698 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3623e353-510a-4e5e-95d9-1a91de0bb0df None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:36.189238 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c938b0-b640-42a4-b2a0-cb3d63bb2222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:36.190301 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c938b0-b640-42a4-b2a0-cb3d63bb2222 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:36.190762 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c938b0-b640-42a4-b2a0-cb3d63bb2222 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:36.191384 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c938b0-b640-42a4-b2a0-cb3d63bb2222 None admin] PATH_INFO: `/v3/users/183f57b7389544378644dd428a134fa4` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:36.208339 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c938b0-b640-42a4-b2a0-cb3d63bb2222 None admin] RBAC: Authorizing `identity:get_user(user_id=183f57b7389544378644dd428a134fa4)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:36.209860 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c938b0-b640-42a4-b2a0-cb3d63bb2222 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:36.212485 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 143/277] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jun 7 14:19:35 2022] GET /identity/v3/users/183f57b7389544378644dd428a134fa4 => generated 293 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:36.213529 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e94b745-3ebf-4873-b567-aa354b01ffa4 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:36.213818 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e94b745-3ebf-4873-b567-aa354b01ffa4 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:36.214099 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e94b745-3ebf-4873-b567-aa354b01ffa4 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:36.215491 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 144/278] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:19:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:19:36.220404 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e94b745-3ebf-4873-b567-aa354b01ffa4 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:36.223433 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:36.342763 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3623e353-510a-4e5e-95d9-1a91de0bb0df None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:36.372305 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e94b745-3ebf-4873-b567-aa354b01ffa4 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:36.402547 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf863267-38a0-4e96-9a78-259f0ef59cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:36.403722 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf863267-38a0-4e96-9a78-259f0ef59cd1 None admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:36.404204 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf863267-38a0-4e96-9a78-259f0ef59cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:36.404743 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf863267-38a0-4e96-9a78-259f0ef59cd1 None admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:36.406073 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf863267-38a0-4e96-9a78-259f0ef59cd1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:36.408797 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf863267-38a0-4e96-9a78-259f0ef59cd1 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:36.412311 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b430728-be7d-45d3-99ed-e015beae1960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:36.412998 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b430728-be7d-45d3-99ed-e015beae1960 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:36.413284 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b430728-be7d-45d3-99ed-e015beae1960 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:36.413562 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b430728-be7d-45d3-99ed-e015beae1960 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:36.414977 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b430728-be7d-45d3-99ed-e015beae1960 None admin] RBAC: Adding query filter params (role.id=3771341b3be641139e752aab690397a4, user.id=183f57b7389544378644dd428a134fa4, scope.system=all) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:19:36.415290 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b430728-be7d-45d3-99ed-e015beae1960 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:36.416748 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b430728-be7d-45d3-99ed-e015beae1960 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:36.423311 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 145/279] 10.222.0.85 () {56 vars in 1308 bytes} [Tue Jun 7 14:19:36 2022] GET /identity/v3/role_assignments?user.id=183f57b7389544378644dd428a134fa4&scope.system=all&role.id=3771341b3be641139e752aab690397a4 => generated 226 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:36.424602 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f42523-fa72-4d7a-8438-db22a70c1422 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:36.424897 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f42523-fa72-4d7a-8438-db22a70c1422 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:36.425177 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f42523-fa72-4d7a-8438-db22a70c1422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:36.430581 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:36.444032 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-42f42523-fa72-4d7a-8438-db22a70c1422 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:19:36.445307 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-42f42523-fa72-4d7a-8438-db22a70c1422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:19:36.530975 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 146/280] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:19:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:19:36.532310 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072f25a6-73bc-4ce3-a271-850d84d855b9 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:36.532591 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072f25a6-73bc-4ce3-a271-850d84d855b9 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:36.533037 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072f25a6-73bc-4ce3-a271-850d84d855b9 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:36.534647 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 147/281] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:19:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:19:36.546842 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b900eb1-34d1-4b3f-af19-fc760f968700 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:36.547246 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b900eb1-34d1-4b3f-af19-fc760f968700 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:36.547487 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b900eb1-34d1-4b3f-af19-fc760f968700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:36.569504 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-8b900eb1-34d1-4b3f-af19-fc760f968700 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:19:36.571677 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-8b900eb1-34d1-4b3f-af19-fc760f968700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:19:36.594225 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 135/282] 10.222.0.85 () {60 vars in 1132 bytes} [Tue Jun 7 14:19:36 2022] POST /identity/v3/users => generated 293 bytes in 438 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:19:36.595388 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf863267-38a0-4e96-9a78-259f0ef59cd1 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:36.721861 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 148/283] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:19:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:19:36.744562 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b900eb1-34d1-4b3f-af19-fc760f968700 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:36.779748 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:36.798599 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf863267-38a0-4e96-9a78-259f0ef59cd1 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:36.862659 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:36.870778 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c15fd79-bf04-4fd0-a4ac-a26db5aa11c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:36.871860 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c15fd79-bf04-4fd0-a4ac-a26db5aa11c9 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:36.872853 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c15fd79-bf04-4fd0-a4ac-a26db5aa11c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:36.873332 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c15fd79-bf04-4fd0-a4ac-a26db5aa11c9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:36.885761 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c15fd79-bf04-4fd0-a4ac-a26db5aa11c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:19:36.893970 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-4c15fd79-bf04-4fd0-a4ac-a26db5aa11c9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 14:19:36.893970 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:19:36.893970 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:19:36.893970 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:19:36.893970 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:19:36.893970 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:19:36.893970 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:19:36.893970 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:19:36.893970 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:19:36.893970 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:19:36.893970 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:19:36.893970 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:19:36.893970 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:19:36.893970 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:19:36.893970 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:19:36.893970 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:19:36.894940 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:19:36.894940 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:19:36.894940 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:19:36.894940 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:19:36.894940 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:19:36.894940 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:19:36.894940 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:19:36.894940 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:19:36.894940 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:19:36.894940 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:19:36.894940 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:19:36.894940 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:19:36.894940 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:19:36.894940 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:19:36.894940 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:19:36.894940 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:19:36.896139 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:19:36.896139 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:19:36.896139 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:19:36.896139 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:19:36.896139 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:19:36.896139 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:19:36.896139 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:19:36.896139 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:19:36.896139 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:19:36.896139 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:19:36.896139 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 14:19:36.896139 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:19:36.896965 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 136/284] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jun 7 14:19:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:19:36.903515 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c15fd79-bf04-4fd0-a4ac-a26db5aa11c9 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:36.927482 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b900eb1-34d1-4b3f-af19-fc760f968700 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:36.977254 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ef983b-b3aa-4480-bbeb-1cd01de2b4b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:36.978287 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ef983b-b3aa-4480-bbeb-1cd01de2b4b0 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:36.978680 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ef983b-b3aa-4480-bbeb-1cd01de2b4b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:36.979027 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ef983b-b3aa-4480-bbeb-1cd01de2b4b0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:36.991701 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ef983b-b3aa-4480-bbeb-1cd01de2b4b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:19:37.001159 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-f6ef983b-b3aa-4480-bbeb-1cd01de2b4b0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 14:19:37.001159 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:19:37.001159 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:19:37.001159 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:19:37.001159 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:19:37.001159 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:19:37.001159 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:19:37.001159 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:19:37.001159 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:19:37.001159 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:19:37.001159 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:19:37.001159 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:19:37.001159 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:19:37.001159 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:19:37.001159 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:19:37.001159 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:19:37.002157 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:19:37.002157 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:19:37.002157 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:19:37.002157 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:19:37.002157 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:19:37.002157 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:19:37.002157 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:19:37.002157 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:19:37.002157 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:19:37.002157 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:19:37.002157 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:19:37.002157 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:19:37.002157 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:19:37.002157 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:19:37.002157 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:19:37.002157 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:19:37.002943 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:19:37.002943 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:19:37.002943 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:19:37.002943 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:19:37.002943 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:19:37.002943 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:19:37.002943 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:19:37.002943 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:19:37.002943 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:19:37.002943 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:19:37.002943 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 14:19:37.002943 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:19:37.002943 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 149/285] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jun 7 14:19:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:19:37.011703 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ef983b-b3aa-4480-bbeb-1cd01de2b4b0 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:37.050690 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c15fd79-bf04-4fd0-a4ac-a26db5aa11c9 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:37.105274 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d866896-26a9-4dc8-9876-0b6bb90b12c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:37.106139 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d866896-26a9-4dc8-9876-0b6bb90b12c3 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:37.106435 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d866896-26a9-4dc8-9876-0b6bb90b12c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:37.106717 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d866896-26a9-4dc8-9876-0b6bb90b12c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:37.107484 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d866896-26a9-4dc8-9876-0b6bb90b12c3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:19:37.107797 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d866896-26a9-4dc8-9876-0b6bb90b12c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:37.110088 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d866896-26a9-4dc8-9876-0b6bb90b12c3 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:37.122268 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 137/286] 10.222.0.85 () {56 vars in 1101 bytes} [Tue Jun 7 14:19:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:19:37.128766 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d866896-26a9-4dc8-9876-0b6bb90b12c3 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:37.149505 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:37.171102 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ef983b-b3aa-4480-bbeb-1cd01de2b4b0 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:37.198625 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:37.217110 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8edcfa-b786-4888-9914-cc1daa809d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:37.218894 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8edcfa-b786-4888-9914-cc1daa809d0a None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:37.219444 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8edcfa-b786-4888-9914-cc1daa809d0a None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:37.220092 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8edcfa-b786-4888-9914-cc1daa809d0a None admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:37.221367 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8edcfa-b786-4888-9914-cc1daa809d0a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:19:37.222107 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8edcfa-b786-4888-9914-cc1daa809d0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:37.225488 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8edcfa-b786-4888-9914-cc1daa809d0a None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:37.237620 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 150/287] 10.222.0.85 () {56 vars in 1101 bytes} [Tue Jun 7 14:19:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:37.246058 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8edcfa-b786-4888-9914-cc1daa809d0a None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:37.278091 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d866896-26a9-4dc8-9876-0b6bb90b12c3 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:37.316388 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a317e641-ba4b-47d0-9326-9eaf87246dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:37.317713 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a317e641-ba4b-47d0-9326-9eaf87246dd8 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:37.318205 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a317e641-ba4b-47d0-9326-9eaf87246dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:37.318703 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a317e641-ba4b-47d0-9326-9eaf87246dd8 None admin] PATH_INFO: `/v3/users/5a235d1390ea42c9b56f16bffef3d4e0` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:37.320398 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a317e641-ba4b-47d0-9326-9eaf87246dd8 None admin] RBAC: Authorizing `identity:get_user(user_id=5a235d1390ea42c9b56f16bffef3d4e0)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:37.322709 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a317e641-ba4b-47d0-9326-9eaf87246dd8 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:37.324726 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 138/288] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jun 7 14:19:37 2022] GET /identity/v3/users/5a235d1390ea42c9b56f16bffef3d4e0 => generated 297 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:37.333622 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a317e641-ba4b-47d0-9326-9eaf87246dd8 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:37.468773 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8edcfa-b786-4888-9914-cc1daa809d0a None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:37.508469 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc8a968-fbe4-484c-ade4-6d8fe0afb05e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:37.509274 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc8a968-fbe4-484c-ade4-6d8fe0afb05e None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:37.509594 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc8a968-fbe4-484c-ade4-6d8fe0afb05e None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:37.510131 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc8a968-fbe4-484c-ade4-6d8fe0afb05e None admin] PATH_INFO: `/v3/users/aa668fea4a7c464eac1ba149b34b69b6` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:37.511818 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc8a968-fbe4-484c-ade4-6d8fe0afb05e None admin] RBAC: Authorizing `identity:get_user(user_id=aa668fea4a7c464eac1ba149b34b69b6)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:37.513241 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:37.514564 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc8a968-fbe4-484c-ade4-6d8fe0afb05e None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:37.517050 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 151/289] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jun 7 14:19:37 2022] GET /identity/v3/users/aa668fea4a7c464eac1ba149b34b69b6 => generated 297 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:37.550777 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc8a968-fbe4-484c-ade4-6d8fe0afb05e None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:37.573805 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a317e641-ba4b-47d0-9326-9eaf87246dd8 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:37.648058 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e620ca0a-3938-4a09-81d9-739964559126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:37.650201 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e620ca0a-3938-4a09-81d9-739964559126 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:37.650709 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e620ca0a-3938-4a09-81d9-739964559126 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:37.651218 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e620ca0a-3938-4a09-81d9-739964559126 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:37.670822 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:37.738116 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc8a968-fbe4-484c-ade4-6d8fe0afb05e None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:37.774431 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc00a536-b1fa-4b58-9ab9-7ea8b903874e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:37.775412 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc00a536-b1fa-4b58-9ab9-7ea8b903874e None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:37.775786 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc00a536-b1fa-4b58-9ab9-7ea8b903874e None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:37.776161 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc00a536-b1fa-4b58-9ab9-7ea8b903874e None admin] PATH_INFO: `/v3/projects/f95de357ccad493984042656b2c0689a` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:37.777536 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc00a536-b1fa-4b58-9ab9-7ea8b903874e None admin] RBAC: Authorizing `identity:get_project(project_id=f95de357ccad493984042656b2c0689a)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:37.779180 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc00a536-b1fa-4b58-9ab9-7ea8b903874e None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:37.781037 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 152/290] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jun 7 14:19:37 2022] GET /identity/v3/projects/f95de357ccad493984042656b2c0689a => generated 302 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:19:37.786681 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc00a536-b1fa-4b58-9ab9-7ea8b903874e None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:37.817348 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e620ca0a-3938-4a09-81d9-739964559126 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:37.819032 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e620ca0a-3938-4a09-81d9-739964559126 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:37.894786 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc00a536-b1fa-4b58-9ab9-7ea8b903874e None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:37.917968 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:37.933849 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1318d9e8-1413-4a1e-b574-c39352a6d47b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:37.934988 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1318d9e8-1413-4a1e-b574-c39352a6d47b None admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:37.935278 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1318d9e8-1413-4a1e-b574-c39352a6d47b None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:37.935556 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1318d9e8-1413-4a1e-b574-c39352a6d47b None admin] PATH_INFO: `/v3/projects/f95de357ccad493984042656b2c0689a/users/aa668fea4a7c464eac1ba149b34b69b6/roles/e2072a2a35d64eec9d32dac7c47c4419` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:37.978818 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1318d9e8-1413-4a1e-b574-c39352a6d47b None admin] RBAC: Authorizing `identity:create_grant(project_id=f95de357ccad493984042656b2c0689a, user_id=aa668fea4a7c464eac1ba149b34b69b6, role_id=e2072a2a35d64eec9d32dac7c47c4419)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:37.978818 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1318d9e8-1413-4a1e-b574-c39352a6d47b None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:37.995040 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:38.033530 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 139/291] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jun 7 14:19:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 700 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 14:19:38.043092 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e620ca0a-3938-4a09-81d9-739964559126 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:38.078684 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 153/292] 10.222.0.85 () {58 vars in 1439 bytes} [Tue Jun 7 14:19:37 2022] PUT /identity/v3/projects/f95de357ccad493984042656b2c0689a/users/aa668fea4a7c464eac1ba149b34b69b6/roles/e2072a2a35d64eec9d32dac7c47c4419 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:19:38.232681 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e620ca0a-3938-4a09-81d9-739964559126 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:38.298877 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54656f2d-776c-4739-b62c-01ad2f1444fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:38.300296 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54656f2d-776c-4739-b62c-01ad2f1444fb None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:38.300828 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54656f2d-776c-4739-b62c-01ad2f1444fb None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:38.301315 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54656f2d-776c-4739-b62c-01ad2f1444fb None admin] PATH_INFO: `/v3/projects/f95de357ccad493984042656b2c0689a` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:38.303584 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54656f2d-776c-4739-b62c-01ad2f1444fb None admin] RBAC: Authorizing `identity:get_project(project_id=f95de357ccad493984042656b2c0689a)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:38.306983 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54656f2d-776c-4739-b62c-01ad2f1444fb None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:38.316553 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 140/293] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jun 7 14:19:38 2022] GET /identity/v3/projects/f95de357ccad493984042656b2c0689a => generated 302 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:19:38.322234 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1318d9e8-1413-4a1e-b574-c39352a6d47b None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:38.461038 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1318d9e8-1413-4a1e-b574-c39352a6d47b None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:38.523319 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efc2356-54db-4fd3-ac6c-26a2571f81d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:38.524627 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efc2356-54db-4fd3-ac6c-26a2571f81d6 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:38.524860 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efc2356-54db-4fd3-ac6c-26a2571f81d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:38.525136 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efc2356-54db-4fd3-ac6c-26a2571f81d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:38.525991 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efc2356-54db-4fd3-ac6c-26a2571f81d6 None admin] RBAC: Adding query filter params (role.id=3771341b3be641139e752aab690397a4, scope.project.id=f95de357ccad493984042656b2c0689a, user.id=5a235d1390ea42c9b56f16bffef3d4e0) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:19:38.526306 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efc2356-54db-4fd3-ac6c-26a2571f81d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:38.527750 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efc2356-54db-4fd3-ac6c-26a2571f81d6 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:38.534448 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 154/294] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jun 7 14:19:38 2022] GET /identity/v3/role_assignments?user.id=5a235d1390ea42c9b56f16bffef3d4e0&scope.project.id=f95de357ccad493984042656b2c0689a&role.id=3771341b3be641139e752aab690397a4 => generated 608 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:39.502022 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5661aa-bdbc-4dd4-9b5e-c9ef6e0fd7cd None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:39.502754 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5661aa-bdbc-4dd4-9b5e-c9ef6e0fd7cd None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:39.502754 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5661aa-bdbc-4dd4-9b5e-c9ef6e0fd7cd None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:39.504025 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 141/295] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:19:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:19:39.510590 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab2dd41-6974-4fe0-b76f-80b921be9196 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:39.510884 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab2dd41-6974-4fe0-b76f-80b921be9196 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:39.511163 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab2dd41-6974-4fe0-b76f-80b921be9196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:39.534929 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-0ab2dd41-6974-4fe0-b76f-80b921be9196 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:19:39.536479 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-0ab2dd41-6974-4fe0-b76f-80b921be9196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:19:39.594182 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:39.640918 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 155/296] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:19:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 14:19:39.676671 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5661aa-bdbc-4dd4-9b5e-c9ef6e0fd7cd None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:39.756240 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:39.849839 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5661aa-bdbc-4dd4-9b5e-c9ef6e0fd7cd None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:39.904594 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aca0321-3010-48c0-8ddf-4976bf18c5f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:39.905471 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aca0321-3010-48c0-8ddf-4976bf18c5f8 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:39.905854 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aca0321-3010-48c0-8ddf-4976bf18c5f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:39.906236 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aca0321-3010-48c0-8ddf-4976bf18c5f8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:39.920180 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aca0321-3010-48c0-8ddf-4976bf18c5f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:19:39.927969 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-7aca0321-3010-48c0-8ddf-4976bf18c5f8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 14:19:39.927969 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:19:39.927969 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:19:39.927969 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:19:39.927969 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:19:39.927969 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:19:39.927969 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:19:39.927969 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:19:39.927969 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:19:39.927969 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:19:39.927969 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:19:39.927969 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:19:39.927969 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:19:39.927969 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:19:39.927969 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:19:39.927969 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:19:39.928891 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:19:39.928891 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:19:39.928891 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:19:39.928891 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:19:39.928891 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:19:39.928891 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:19:39.928891 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:19:39.928891 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:19:39.928891 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:19:39.928891 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:19:39.928891 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:19:39.928891 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:19:39.928891 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:19:39.928891 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:19:39.928891 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:19:39.928891 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:19:39.929669 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:19:39.929669 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:19:39.929669 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:19:39.929669 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:19:39.929669 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:19:39.929669 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:19:39.929669 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:19:39.929669 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:19:39.929669 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:19:39.929669 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:19:39.929669 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 14:19:39.929669 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:19:39.929669 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 142/297] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jun 7 14:19:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:19:39.936706 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab2dd41-6974-4fe0-b76f-80b921be9196 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:39.943094 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef488400-20c4-4ac8-8855-0c976faec31c None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:39.943429 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef488400-20c4-4ac8-8855-0c976faec31c None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:39.943893 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef488400-20c4-4ac8-8855-0c976faec31c None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:39.951727 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 143/298] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:19:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 14:19:39.959807 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256cb817-eef0-47c7-b39b-9d36fd943b2d None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:39.959807 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256cb817-eef0-47c7-b39b-9d36fd943b2d None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:39.959807 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256cb817-eef0-47c7-b39b-9d36fd943b2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:39.984721 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-256cb817-eef0-47c7-b39b-9d36fd943b2d None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:19:39.986739 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-256cb817-eef0-47c7-b39b-9d36fd943b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:19:40.069688 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab2dd41-6974-4fe0-b76f-80b921be9196 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:40.081710 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 144/299] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:19:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 14:19:40.108529 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256cb817-eef0-47c7-b39b-9d36fd943b2d None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:40.127784 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81217346-ba81-449e-8f39-1d98addcb5bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:40.129860 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81217346-ba81-449e-8f39-1d98addcb5bd None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:40.130644 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81217346-ba81-449e-8f39-1d98addcb5bd None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:40.131702 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81217346-ba81-449e-8f39-1d98addcb5bd None admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:40.132723 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81217346-ba81-449e-8f39-1d98addcb5bd None admin] RBAC: Adding query filter params (name=member) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:19:40.133669 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81217346-ba81-449e-8f39-1d98addcb5bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:40.135276 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81217346-ba81-449e-8f39-1d98addcb5bd None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:40.153451 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 156/300] 10.222.0.85 () {56 vars in 1101 bytes} [Tue Jun 7 14:19:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:19:40.160121 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81217346-ba81-449e-8f39-1d98addcb5bd None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:40.184745 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:40.197893 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:40.270622 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81217346-ba81-449e-8f39-1d98addcb5bd None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:40.295290 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256cb817-eef0-47c7-b39b-9d36fd943b2d None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:40.316355 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b7e3ad-ab69-4ac7-acd5-38b4f11c2997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:40.317295 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b7e3ad-ab69-4ac7-acd5-38b4f11c2997 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:40.317638 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b7e3ad-ab69-4ac7-acd5-38b4f11c2997 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:40.317914 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b7e3ad-ab69-4ac7-acd5-38b4f11c2997 None admin] PATH_INFO: `/v3/users/183f57b7389544378644dd428a134fa4` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:40.319785 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b7e3ad-ab69-4ac7-acd5-38b4f11c2997 None admin] RBAC: Authorizing `identity:get_user(user_id=183f57b7389544378644dd428a134fa4)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:40.322020 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b7e3ad-ab69-4ac7-acd5-38b4f11c2997 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:40.323920 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 157/301] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jun 7 14:19:40 2022] GET /identity/v3/users/183f57b7389544378644dd428a134fa4 => generated 293 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:19:40.325117 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3e38e9-9766-4294-b22e-89dd3ebf3d60 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:40.325416 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3e38e9-9766-4294-b22e-89dd3ebf3d60 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:40.325699 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3e38e9-9766-4294-b22e-89dd3ebf3d60 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:40.327125 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 158/302] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:19:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:19:40.331976 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3e38e9-9766-4294-b22e-89dd3ebf3d60 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:40.338899 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3471b6f-e587-4df9-9e51-4fb02a8ac100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:40.339970 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3471b6f-e587-4df9-9e51-4fb02a8ac100 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:40.340467 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3471b6f-e587-4df9-9e51-4fb02a8ac100 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:40.340917 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3471b6f-e587-4df9-9e51-4fb02a8ac100 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:40.353375 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3471b6f-e587-4df9-9e51-4fb02a8ac100 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:19:40.361439 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-d3471b6f-e587-4df9-9e51-4fb02a8ac100 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 14:19:40.361439 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:19:40.361439 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:19:40.361439 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:19:40.361439 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:19:40.361439 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:19:40.361439 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:19:40.361439 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:19:40.361439 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:19:40.361439 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:19:40.361439 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:19:40.361439 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:19:40.361439 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:19:40.361439 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:19:40.361439 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:19:40.361439 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:19:40.362342 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:19:40.362342 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:19:40.362342 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:19:40.362342 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:19:40.362342 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:19:40.362342 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:19:40.362342 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:19:40.362342 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:19:40.362342 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:19:40.362342 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:19:40.362342 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:19:40.362342 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:19:40.362342 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:19:40.362342 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:19:40.362342 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:19:40.362342 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:19:40.363126 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:19:40.363126 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:19:40.363126 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:19:40.363126 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:19:40.363126 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:19:40.363126 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:19:40.363126 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:19:40.363126 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:19:40.363126 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:19:40.363126 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:19:40.363126 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 14:19:40.363126 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:19:40.371296 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 145/303] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jun 7 14:19:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:19:40.371296 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b27beff-ce3e-486b-997f-02bb6fb27216 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:40.371296 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b27beff-ce3e-486b-997f-02bb6fb27216 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:40.371296 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b27beff-ce3e-486b-997f-02bb6fb27216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:40.388968 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-0b27beff-ce3e-486b-997f-02bb6fb27216 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:19:40.390271 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-0b27beff-ce3e-486b-997f-02bb6fb27216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:19:40.480150 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3e38e9-9766-4294-b22e-89dd3ebf3d60 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:40.482935 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:40.497026 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 146/304] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:19:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:19:40.502305 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b27beff-ce3e-486b-997f-02bb6fb27216 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:40.529883 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abac4873-22b8-461b-b34e-2d2a9222cb8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:40.530881 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abac4873-22b8-461b-b34e-2d2a9222cb8e None admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:40.531127 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abac4873-22b8-461b-b34e-2d2a9222cb8e None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:40.531495 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abac4873-22b8-461b-b34e-2d2a9222cb8e None admin] PATH_INFO: `/v3/system/users/183f57b7389544378644dd428a134fa4/roles/3771341b3be641139e752aab690397a4` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:40.533431 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abac4873-22b8-461b-b34e-2d2a9222cb8e None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=183f57b7389544378644dd428a134fa4, role_id=3771341b3be641139e752aab690397a4)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:40.535093 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abac4873-22b8-461b-b34e-2d2a9222cb8e None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:40.543024 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:40.618150 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 159/305] 10.222.0.85 () {58 vars in 1334 bytes} [Tue Jun 7 14:19:40 2022] PUT /identity/v3/system/users/183f57b7389544378644dd428a134fa4/roles/3771341b3be641139e752aab690397a4 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:19:40.621810 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abac4873-22b8-461b-b34e-2d2a9222cb8e None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:40.640358 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b27beff-ce3e-486b-997f-02bb6fb27216 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:40.688207 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e0aa7f-4de2-4288-b8cd-2bf26d381ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:40.689558 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e0aa7f-4de2-4288-b8cd-2bf26d381ba0 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:40.690201 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e0aa7f-4de2-4288-b8cd-2bf26d381ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:40.690765 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e0aa7f-4de2-4288-b8cd-2bf26d381ba0 None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:40.691979 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e0aa7f-4de2-4288-b8cd-2bf26d381ba0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:19:40.692698 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e0aa7f-4de2-4288-b8cd-2bf26d381ba0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:40.694517 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e0aa7f-4de2-4288-b8cd-2bf26d381ba0 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:40.713380 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 147/306] 10.222.0.85 () {56 vars in 1101 bytes} [Tue Jun 7 14:19:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:40.718019 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e0aa7f-4de2-4288-b8cd-2bf26d381ba0 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:40.780256 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abac4873-22b8-461b-b34e-2d2a9222cb8e None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:40.832953 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9110d17b-38c3-426a-a90b-91805c06beeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:40.833937 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9110d17b-38c3-426a-a90b-91805c06beeb None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:40.834295 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9110d17b-38c3-426a-a90b-91805c06beeb None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:40.834678 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9110d17b-38c3-426a-a90b-91805c06beeb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:40.836589 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9110d17b-38c3-426a-a90b-91805c06beeb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:40.838814 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9110d17b-38c3-426a-a90b-91805c06beeb None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:40.841673 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 160/307] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:19:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:40.848991 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9110d17b-38c3-426a-a90b-91805c06beeb None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:40.858970 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:40.874378 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e0aa7f-4de2-4288-b8cd-2bf26d381ba0 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:40.902771 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:40.928880 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d8ddf1-51b7-4a1e-9239-b9ee36ec29ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:40.930682 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d8ddf1-51b7-4a1e-9239-b9ee36ec29ac None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:40.931337 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d8ddf1-51b7-4a1e-9239-b9ee36ec29ac None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:40.931761 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d8ddf1-51b7-4a1e-9239-b9ee36ec29ac None admin] PATH_INFO: `/v3/users/6093a023afe9438ca266366fb816c855` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:40.961944 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d8ddf1-51b7-4a1e-9239-b9ee36ec29ac None admin] RBAC: Authorizing `identity:get_user(user_id=6093a023afe9438ca266366fb816c855)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:40.965425 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d8ddf1-51b7-4a1e-9239-b9ee36ec29ac None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:40.968208 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 148/308] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jun 7 14:19:40 2022] GET /identity/v3/users/6093a023afe9438ca266366fb816c855 => generated 293 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:19:40.972018 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d8ddf1-51b7-4a1e-9239-b9ee36ec29ac None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:41.006412 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9110d17b-38c3-426a-a90b-91805c06beeb None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:41.055996 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9057b4-2e22-4ac0-a70a-38ab380f05e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:41.056865 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9057b4-2e22-4ac0-a70a-38ab380f05e9 None admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:41.057194 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9057b4-2e22-4ac0-a70a-38ab380f05e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:41.057622 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9057b4-2e22-4ac0-a70a-38ab380f05e9 None admin] PATH_INFO: `/v3/groups` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:41.058534 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9057b4-2e22-4ac0-a70a-38ab380f05e9 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:41.060246 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9057b4-2e22-4ac0-a70a-38ab380f05e9 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:41.099083 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d8ddf1-51b7-4a1e-9239-b9ee36ec29ac None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:41.136533 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183d5df9-1002-4b04-b691-07942d8df9af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:41.137125 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183d5df9-1002-4b04-b691-07942d8df9af None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:41.137408 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183d5df9-1002-4b04-b691-07942d8df9af None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:41.137687 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183d5df9-1002-4b04-b691-07942d8df9af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:41.138521 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183d5df9-1002-4b04-b691-07942d8df9af None admin] RBAC: Adding query filter params (role.id=e2072a2a35d64eec9d32dac7c47c4419, user.id=6093a023afe9438ca266366fb816c855, scope.system=all) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:19:41.138835 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183d5df9-1002-4b04-b691-07942d8df9af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:41.140308 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183d5df9-1002-4b04-b691-07942d8df9af None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:41.160439 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 149/309] 10.222.0.85 () {56 vars in 1308 bytes} [Tue Jun 7 14:19:40 2022] GET /identity/v3/role_assignments?user.id=6093a023afe9438ca266366fb816c855&scope.system=all&role.id=e2072a2a35d64eec9d32dac7c47c4419 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:41.160687 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 161/310] 10.222.0.85 () {60 vars in 1134 bytes} [Tue Jun 7 14:19:40 2022] POST /identity/v3/groups => generated 229 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:19:41.160823 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1892732a-c220-481d-9561-4d81bf25545d None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:41.160823 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1892732a-c220-481d-9561-4d81bf25545d None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:41.160823 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1892732a-c220-481d-9561-4d81bf25545d None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:41.160823 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 150/311] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:19:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:19:41.164681 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddd95d8-dd5a-4261-93a7-af6be01dc26b None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:41.164996 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddd95d8-dd5a-4261-93a7-af6be01dc26b None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:41.165253 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddd95d8-dd5a-4261-93a7-af6be01dc26b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:41.196029 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-bddd95d8-dd5a-4261-93a7-af6be01dc26b None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:19:41.198470 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-bddd95d8-dd5a-4261-93a7-af6be01dc26b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:19:41.303620 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 162/312] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:19:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:19:41.333183 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1892732a-c220-481d-9561-4d81bf25545d None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:41.355280 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:41.485261 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1892732a-c220-481d-9561-4d81bf25545d None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:41.526241 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f0d009-01f5-4574-8d29-cfa7c0e84559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:41.527135 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f0d009-01f5-4574-8d29-cfa7c0e84559 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:41.527483 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f0d009-01f5-4574-8d29-cfa7c0e84559 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:41.527821 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f0d009-01f5-4574-8d29-cfa7c0e84559 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:41.538717 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f0d009-01f5-4574-8d29-cfa7c0e84559 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:19:41.547459 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-57f0d009-01f5-4574-8d29-cfa7c0e84559 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 14:19:41.547459 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:19:41.547459 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:19:41.547459 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:19:41.547459 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:19:41.547459 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:19:41.547459 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:19:41.547459 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:19:41.547459 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:19:41.547459 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:19:41.547459 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:19:41.547459 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:19:41.547459 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:19:41.547459 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:19:41.547459 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:19:41.547459 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:19:41.548407 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:19:41.548407 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:19:41.548407 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:19:41.548407 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:19:41.548407 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:19:41.548407 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:19:41.548407 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:19:41.548407 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:19:41.548407 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:19:41.548407 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:19:41.548407 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:19:41.548407 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:19:41.548407 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:19:41.548407 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:19:41.548407 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:19:41.548407 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:19:41.549197 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:19:41.549197 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:19:41.549197 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:19:41.549197 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:19:41.549197 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:19:41.549197 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:19:41.549197 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:19:41.549197 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:19:41.549197 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:19:41.549197 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:19:41.549197 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 14:19:41.549197 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:19:41.549197 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 151/313] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jun 7 14:19:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:19:41.554991 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddd95d8-dd5a-4261-93a7-af6be01dc26b None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:41.588106 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:41.664043 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddd95d8-dd5a-4261-93a7-af6be01dc26b None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:41.710503 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb5b386-2409-43a0-aab2-c41d970bbacb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:41.712084 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb5b386-2409-43a0-aab2-c41d970bbacb None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:41.712591 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb5b386-2409-43a0-aab2-c41d970bbacb None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:41.713045 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb5b386-2409-43a0-aab2-c41d970bbacb None admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:41.714228 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb5b386-2409-43a0-aab2-c41d970bbacb None admin] RBAC: Adding query filter params (name=reader) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:19:41.714708 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb5b386-2409-43a0-aab2-c41d970bbacb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:41.716884 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb5b386-2409-43a0-aab2-c41d970bbacb None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:41.726439 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 163/314] 10.222.0.85 () {56 vars in 1101 bytes} [Tue Jun 7 14:19:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:41.733809 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f0d009-01f5-4574-8d29-cfa7c0e84559 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:41.819231 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:41.852978 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f0d009-01f5-4574-8d29-cfa7c0e84559 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:41.889919 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8833a80-c821-4d36-8db5-caba74ac0940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:41.890814 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8833a80-c821-4d36-8db5-caba74ac0940 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:41.891104 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8833a80-c821-4d36-8db5-caba74ac0940 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:41.891388 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8833a80-c821-4d36-8db5-caba74ac0940 None admin] PATH_INFO: `/v3/users/aa668fea4a7c464eac1ba149b34b69b6` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:41.892691 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8833a80-c821-4d36-8db5-caba74ac0940 None admin] RBAC: Authorizing `identity:get_user(user_id=aa668fea4a7c464eac1ba149b34b69b6)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:41.894182 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8833a80-c821-4d36-8db5-caba74ac0940 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:41.896151 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 152/315] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jun 7 14:19:41 2022] GET /identity/v3/users/aa668fea4a7c464eac1ba149b34b69b6 => generated 297 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:41.902052 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb5b386-2409-43a0-aab2-c41d970bbacb None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:42.012844 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:42.030210 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb5b386-2409-43a0-aab2-c41d970bbacb None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:42.080290 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eb91ba-5b59-441e-81e4-747241eeedcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:42.080909 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eb91ba-5b59-441e-81e4-747241eeedcf None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:42.081194 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eb91ba-5b59-441e-81e4-747241eeedcf None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:42.081471 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eb91ba-5b59-441e-81e4-747241eeedcf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:42.181168 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76eb91ba-5b59-441e-81e4-747241eeedcf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:42.183838 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76eb91ba-5b59-441e-81e4-747241eeedcf None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:42.328966 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:42.343642 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 164/316] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jun 7 14:19:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 14:19:42.350921 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8833a80-c821-4d36-8db5-caba74ac0940 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:42.502620 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8833a80-c821-4d36-8db5-caba74ac0940 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:42.538391 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1cc9dc-0526-4f5a-a2ab-5b1ef5f19320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:42.539802 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1cc9dc-0526-4f5a-a2ab-5b1ef5f19320 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:42.540094 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1cc9dc-0526-4f5a-a2ab-5b1ef5f19320 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:42.540391 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1cc9dc-0526-4f5a-a2ab-5b1ef5f19320 None admin] PATH_INFO: `/v3/projects/f95de357ccad493984042656b2c0689a` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:42.542095 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1cc9dc-0526-4f5a-a2ab-5b1ef5f19320 None admin] RBAC: Authorizing `identity:get_project(project_id=f95de357ccad493984042656b2c0689a)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:42.543628 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1cc9dc-0526-4f5a-a2ab-5b1ef5f19320 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:42.545620 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 153/317] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jun 7 14:19:42 2022] GET /identity/v3/projects/f95de357ccad493984042656b2c0689a => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:42.552524 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eb91ba-5b59-441e-81e4-747241eeedcf None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:42.685986 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eb91ba-5b59-441e-81e4-747241eeedcf None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:42.737199 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b81b5e0-9b37-4527-bbd9-45a8f86bdab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:42.737866 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b81b5e0-9b37-4527-bbd9-45a8f86bdab8 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:42.738149 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b81b5e0-9b37-4527-bbd9-45a8f86bdab8 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:42.738427 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b81b5e0-9b37-4527-bbd9-45a8f86bdab8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:42.739275 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b81b5e0-9b37-4527-bbd9-45a8f86bdab8 None admin] RBAC: Adding query filter params (role.id=e2072a2a35d64eec9d32dac7c47c4419, scope.project.id=f95de357ccad493984042656b2c0689a, user.id=aa668fea4a7c464eac1ba149b34b69b6) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:19:42.739591 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b81b5e0-9b37-4527-bbd9-45a8f86bdab8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:42.741082 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b81b5e0-9b37-4527-bbd9-45a8f86bdab8 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:42.751410 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 165/318] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jun 7 14:19:42 2022] GET /identity/v3/role_assignments?user.id=aa668fea4a7c464eac1ba149b34b69b6&scope.project.id=f95de357ccad493984042656b2c0689a&role.id=e2072a2a35d64eec9d32dac7c47c4419 => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:19:43.331428 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c610990c-e7e6-473a-bde5-3bf49f559e79 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:43.331773 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c610990c-e7e6-473a-bde5-3bf49f559e79 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:43.332125 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c610990c-e7e6-473a-bde5-3bf49f559e79 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:43.334016 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 154/319] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:19:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:19:43.341440 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1899b6bd-1ce9-4a07-ae42-406865796de4 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:43.341782 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1899b6bd-1ce9-4a07-ae42-406865796de4 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:43.342168 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1899b6bd-1ce9-4a07-ae42-406865796de4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:43.364951 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-1899b6bd-1ce9-4a07-ae42-406865796de4 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:19:43.366164 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-1899b6bd-1ce9-4a07-ae42-406865796de4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:19:43.437486 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:43.502426 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 166/320] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:19:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:19:43.535411 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c610990c-e7e6-473a-bde5-3bf49f559e79 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:43.551868 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:43.666671 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c610990c-e7e6-473a-bde5-3bf49f559e79 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:43.714328 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3666deb3-52cf-4647-b755-9c7a04b0232c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:43.715148 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3666deb3-52cf-4647-b755-9c7a04b0232c None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:43.716878 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3666deb3-52cf-4647-b755-9c7a04b0232c None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:43.716878 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3666deb3-52cf-4647-b755-9c7a04b0232c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:43.736890 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3666deb3-52cf-4647-b755-9c7a04b0232c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:19:43.745795 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-3666deb3-52cf-4647-b755-9c7a04b0232c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 14:19:43.745795 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:19:43.745795 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:19:43.745795 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:19:43.745795 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:19:43.745795 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:19:43.745795 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:19:43.745795 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:19:43.745795 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:19:43.745795 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:19:43.745795 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:19:43.745795 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:19:43.745795 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:19:43.745795 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:19:43.745795 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:19:43.745795 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:19:43.747096 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:19:43.747096 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:19:43.747096 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:19:43.747096 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:19:43.747096 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:19:43.747096 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:19:43.747096 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:19:43.747096 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:19:43.747096 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:19:43.747096 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:19:43.747096 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:19:43.747096 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:19:43.747096 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:19:43.747096 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:19:43.747096 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:19:43.747096 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:19:43.748321 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:19:43.748321 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:19:43.748321 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:19:43.748321 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:19:43.748321 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:19:43.748321 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:19:43.748321 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:19:43.748321 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:19:43.748321 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:19:43.748321 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:19:43.748321 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 14:19:43.748321 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:19:43.748321 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 155/321] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jun 7 14:19:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:19:43.756644 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1899b6bd-1ce9-4a07-ae42-406865796de4 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:43.802128 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cbf5b4-ab0b-4734-a29d-80ae7931a601 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:43.802531 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cbf5b4-ab0b-4734-a29d-80ae7931a601 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:43.802684 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cbf5b4-ab0b-4734-a29d-80ae7931a601 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:43.804194 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 156/322] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:19:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 14:19:43.809851 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b10bcc-a47a-4355-859d-17e1233981b0 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:43.810137 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b10bcc-a47a-4355-859d-17e1233981b0 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:43.810421 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b10bcc-a47a-4355-859d-17e1233981b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:43.828906 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-64b10bcc-a47a-4355-859d-17e1233981b0 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:19:43.830118 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-64b10bcc-a47a-4355-859d-17e1233981b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:19:43.903601 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1899b6bd-1ce9-4a07-ae42-406865796de4 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:43.914143 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:43.919762 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 157/323] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:19:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 14:19:43.921946 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa39c2d6-6724-4b4d-ac9c-86e0dfe8342e None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:43.922236 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa39c2d6-6724-4b4d-ac9c-86e0dfe8342e None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:43.922517 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa39c2d6-6724-4b4d-ac9c-86e0dfe8342e None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:43.923852 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 158/324] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:19:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:19:43.931651 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d802eba8-3eef-404b-9e3f-a173b94857e6 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:43.931874 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d802eba8-3eef-404b-9e3f-a173b94857e6 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:43.932467 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d802eba8-3eef-404b-9e3f-a173b94857e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:43.951651 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cf00c7-d340-4d45-88fd-5a5b23175c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:43.952328 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cf00c7-d340-4d45-88fd-5a5b23175c57 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:43.952648 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cf00c7-d340-4d45-88fd-5a5b23175c57 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:43.952931 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cf00c7-d340-4d45-88fd-5a5b23175c57 None admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:43.953689 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cf00c7-d340-4d45-88fd-5a5b23175c57 None admin] RBAC: Adding query filter params (name=member) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:19:43.954001 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cf00c7-d340-4d45-88fd-5a5b23175c57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:43.957024 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-d802eba8-3eef-404b-9e3f-a173b94857e6 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:19:43.957162 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cf00c7-d340-4d45-88fd-5a5b23175c57 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:43.960040 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-d802eba8-3eef-404b-9e3f-a173b94857e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:19:43.973174 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 167/325] 10.222.0.85 () {56 vars in 1101 bytes} [Tue Jun 7 14:19:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:19:43.973174 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cf00c7-d340-4d45-88fd-5a5b23175c57 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:44.028225 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:44.040626 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 159/326] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:19:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 14:19:44.042471 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d802eba8-3eef-404b-9e3f-a173b94857e6 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:44.096202 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cf00c7-d340-4d45-88fd-5a5b23175c57 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:44.143150 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993ecd8a-6b95-4327-a74c-b8a33f8841e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:44.143956 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993ecd8a-6b95-4327-a74c-b8a33f8841e5 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:44.144244 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993ecd8a-6b95-4327-a74c-b8a33f8841e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:44.144553 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993ecd8a-6b95-4327-a74c-b8a33f8841e5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:44.156612 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993ecd8a-6b95-4327-a74c-b8a33f8841e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:19:44.164914 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-993ecd8a-6b95-4327-a74c-b8a33f8841e5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 14:19:44.164914 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:19:44.164914 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:19:44.164914 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:19:44.164914 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:19:44.164914 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:19:44.164914 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:19:44.164914 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:19:44.164914 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:19:44.164914 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:19:44.164914 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:19:44.164914 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:19:44.164914 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:19:44.164914 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:19:44.164914 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:19:44.164914 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:19:44.165874 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:19:44.165874 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:19:44.165874 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:19:44.165874 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:19:44.165874 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:19:44.165874 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:19:44.165874 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:19:44.165874 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:19:44.165874 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:19:44.165874 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:19:44.165874 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:19:44.165874 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:19:44.165874 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:19:44.165874 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:19:44.165874 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:19:44.165874 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:19:44.166644 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:19:44.166644 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:19:44.166644 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:19:44.166644 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:19:44.166644 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:19:44.166644 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:19:44.166644 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:19:44.166644 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:19:44.166644 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:19:44.166644 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:19:44.166644 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 14:19:44.166644 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:19:44.166644 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 168/327] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jun 7 14:19:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:19:44.167277 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993ecd8a-6b95-4327-a74c-b8a33f8841e5 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:44.178772 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d802eba8-3eef-404b-9e3f-a173b94857e6 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:44.214177 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:44.229488 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707c1a09-ae3d-469b-8bc2-bad048ff5767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:44.230379 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707c1a09-ae3d-469b-8bc2-bad048ff5767 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:44.230666 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707c1a09-ae3d-469b-8bc2-bad048ff5767 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:44.230945 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707c1a09-ae3d-469b-8bc2-bad048ff5767 None admin] PATH_INFO: `/v3/users/183f57b7389544378644dd428a134fa4` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:44.232174 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707c1a09-ae3d-469b-8bc2-bad048ff5767 None admin] RBAC: Authorizing `identity:get_user(user_id=183f57b7389544378644dd428a134fa4)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:44.233687 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707c1a09-ae3d-469b-8bc2-bad048ff5767 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:44.241266 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 160/328] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jun 7 14:19:44 2022] GET /identity/v3/users/183f57b7389544378644dd428a134fa4 => generated 293 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:44.241266 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707c1a09-ae3d-469b-8bc2-bad048ff5767 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:44.296057 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993ecd8a-6b95-4327-a74c-b8a33f8841e5 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:44.299088 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:44.332710 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7ef6ae-f4e9-44c0-9e16-18b6886c5e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:44.334107 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7ef6ae-f4e9-44c0-9e16-18b6886c5e10 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:44.334517 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7ef6ae-f4e9-44c0-9e16-18b6886c5e10 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:44.335086 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7ef6ae-f4e9-44c0-9e16-18b6886c5e10 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:44.336895 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7ef6ae-f4e9-44c0-9e16-18b6886c5e10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:44.338413 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7ef6ae-f4e9-44c0-9e16-18b6886c5e10 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:44.340408 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 169/329] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:19:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:44.340844 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7ef6ae-f4e9-44c0-9e16-18b6886c5e10 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:44.349662 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707c1a09-ae3d-469b-8bc2-bad048ff5767 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:44.395799 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47585d21-05f5-42e4-a800-d622ae5b7b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:44.396506 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47585d21-05f5-42e4-a800-d622ae5b7b71 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:44.396799 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47585d21-05f5-42e4-a800-d622ae5b7b71 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:44.397084 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47585d21-05f5-42e4-a800-d622ae5b7b71 None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:44.397844 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47585d21-05f5-42e4-a800-d622ae5b7b71 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:19:44.398159 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47585d21-05f5-42e4-a800-d622ae5b7b71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:44.400095 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47585d21-05f5-42e4-a800-d622ae5b7b71 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:44.409115 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 161/330] 10.222.0.85 () {56 vars in 1101 bytes} [Tue Jun 7 14:19:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:44.409991 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47585d21-05f5-42e4-a800-d622ae5b7b71 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:44.455863 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7ef6ae-f4e9-44c0-9e16-18b6886c5e10 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:44.483327 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:44.513505 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7128a39-5545-4c37-870e-fe939c6bef71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:44.514293 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7128a39-5545-4c37-870e-fe939c6bef71 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:44.514590 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7128a39-5545-4c37-870e-fe939c6bef71 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:44.514867 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7128a39-5545-4c37-870e-fe939c6bef71 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:44.516051 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7128a39-5545-4c37-870e-fe939c6bef71 None admin] RBAC: Adding query filter params (role.id=3771341b3be641139e752aab690397a4, user.id=183f57b7389544378644dd428a134fa4, scope.system=all) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:19:44.516051 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7128a39-5545-4c37-870e-fe939c6bef71 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:44.517530 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7128a39-5545-4c37-870e-fe939c6bef71 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:44.533987 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47585d21-05f5-42e4-a800-d622ae5b7b71 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:44.545101 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 170/331] 10.222.0.85 () {56 vars in 1308 bytes} [Tue Jun 7 14:19:44 2022] GET /identity/v3/role_assignments?user.id=183f57b7389544378644dd428a134fa4&scope.system=all&role.id=3771341b3be641139e752aab690397a4 => generated 510 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:44.545101 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7128a39-5545-4c37-870e-fe939c6bef71 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:44.590914 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:44.593064 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09187758-bda3-42b3-8ba3-0db3a73007f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:44.593782 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09187758-bda3-42b3-8ba3-0db3a73007f1 None admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:44.593782 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09187758-bda3-42b3-8ba3-0db3a73007f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:44.593997 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09187758-bda3-42b3-8ba3-0db3a73007f1 None admin] PATH_INFO: `/v3/groups` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:44.600675 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09187758-bda3-42b3-8ba3-0db3a73007f1 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:44.600675 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09187758-bda3-42b3-8ba3-0db3a73007f1 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:44.695527 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7128a39-5545-4c37-870e-fe939c6bef71 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:44.719558 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 162/332] 10.222.0.85 () {60 vars in 1134 bytes} [Tue Jun 7 14:19:44 2022] POST /identity/v3/groups => generated 226 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:19:44.775414 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfcd9da-80a6-4085-9fe9-b3b970f6faf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:44.776352 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfcd9da-80a6-4085-9fe9-b3b970f6faf8 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:44.776677 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfcd9da-80a6-4085-9fe9-b3b970f6faf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:44.776947 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfcd9da-80a6-4085-9fe9-b3b970f6faf8 None admin] PATH_INFO: `/v3/users/6093a023afe9438ca266366fb816c855` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:44.778349 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfcd9da-80a6-4085-9fe9-b3b970f6faf8 None admin] RBAC: Authorizing `identity:get_user(user_id=6093a023afe9438ca266366fb816c855)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:44.779841 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfcd9da-80a6-4085-9fe9-b3b970f6faf8 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:44.781638 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 171/333] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jun 7 14:19:44 2022] GET /identity/v3/users/6093a023afe9438ca266366fb816c855 => generated 293 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:44.787647 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09187758-bda3-42b3-8ba3-0db3a73007f1 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:44.944223 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09187758-bda3-42b3-8ba3-0db3a73007f1 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:44.965148 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:44.983058 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e20c9a-cd27-4512-baf9-8fb9929f2e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:44.983844 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e20c9a-cd27-4512-baf9-8fb9929f2e07 None admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:44.984129 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e20c9a-cd27-4512-baf9-8fb9929f2e07 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:44.984428 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e20c9a-cd27-4512-baf9-8fb9929f2e07 None admin] PATH_INFO: `/v3/system/users/6093a023afe9438ca266366fb816c855/roles/e2072a2a35d64eec9d32dac7c47c4419` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:44.986033 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e20c9a-cd27-4512-baf9-8fb9929f2e07 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=6093a023afe9438ca266366fb816c855, role_id=e2072a2a35d64eec9d32dac7c47c4419)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:44.987409 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e20c9a-cd27-4512-baf9-8fb9929f2e07 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:45.067556 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 163/334] 10.222.0.85 () {58 vars in 1334 bytes} [Tue Jun 7 14:19:44 2022] PUT /identity/v3/system/users/6093a023afe9438ca266366fb816c855/roles/e2072a2a35d64eec9d32dac7c47c4419 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:19:47.809431 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c96cc25-6745-4767-8c7e-4e9d835a2f7f None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:47.810455 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c96cc25-6745-4767-8c7e-4e9d835a2f7f None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:47.810979 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c96cc25-6745-4767-8c7e-4e9d835a2f7f None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:47.812944 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 172/335] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:19:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:19:47.818434 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee8ac1c-a91d-4956-8e9d-4d0350d2d953 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:47.819372 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee8ac1c-a91d-4956-8e9d-4d0350d2d953 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:47.819852 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee8ac1c-a91d-4956-8e9d-4d0350d2d953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:47.845437 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-aee8ac1c-a91d-4956-8e9d-4d0350d2d953 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:19:47.847021 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-aee8ac1c-a91d-4956-8e9d-4d0350d2d953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:19:47.964479 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 164/336] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:19:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:19:47.997020 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c96cc25-6745-4767-8c7e-4e9d835a2f7f None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:48.145810 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:48.153038 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c96cc25-6745-4767-8c7e-4e9d835a2f7f None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:48.174463 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb99312-6fd6-4937-8176-1f4d8cd86777 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:48.174793 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb99312-6fd6-4937-8176-1f4d8cd86777 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:48.175027 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb99312-6fd6-4937-8176-1f4d8cd86777 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:48.176696 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 165/337] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:19:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:19:48.185004 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990154db-e494-45fe-9b86-a67cd35c4463 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:48.185295 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990154db-e494-45fe-9b86-a67cd35c4463 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:48.185579 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990154db-e494-45fe-9b86-a67cd35c4463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:48.205510 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74100a8-e7f8-4eb7-8747-8aa9aef08138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:48.206652 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74100a8-e7f8-4eb7-8747-8aa9aef08138 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:48.207097 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74100a8-e7f8-4eb7-8747-8aa9aef08138 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:48.207533 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74100a8-e7f8-4eb7-8747-8aa9aef08138 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:48.223064 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-990154db-e494-45fe-9b86-a67cd35c4463 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:19:48.225219 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-990154db-e494-45fe-9b86-a67cd35c4463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:19:48.236884 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74100a8-e7f8-4eb7-8747-8aa9aef08138 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:19:48.245373 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-f74100a8-e7f8-4eb7-8747-8aa9aef08138 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 14:19:48.245373 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:19:48.245373 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:19:48.245373 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:19:48.245373 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:19:48.245373 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:19:48.245373 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:19:48.245373 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:19:48.245373 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:19:48.245373 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:19:48.245373 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:19:48.245373 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:19:48.245373 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:19:48.245373 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:19:48.245373 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:19:48.245373 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:19:48.248175 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:19:48.248175 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:19:48.248175 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:19:48.248175 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:19:48.248175 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:19:48.248175 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:19:48.248175 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:19:48.248175 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:19:48.248175 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:19:48.248175 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:19:48.248175 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:19:48.248175 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:19:48.248175 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:19:48.248175 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:19:48.248175 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:19:48.248175 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:19:48.248976 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:19:48.248976 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:19:48.248976 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:19:48.248976 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:19:48.248976 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:19:48.248976 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:19:48.248976 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:19:48.248976 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:19:48.248976 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:19:48.248976 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:19:48.248976 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 14:19:48.248976 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:19:48.261761 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 173/338] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jun 7 14:19:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:19:48.266175 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74100a8-e7f8-4eb7-8747-8aa9aef08138 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:48.367237 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 166/339] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:19:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:19:48.368834 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dbe3d3-3201-4bde-b76f-a845d9bad3f1 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:48.369136 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dbe3d3-3201-4bde-b76f-a845d9bad3f1 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:48.369683 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dbe3d3-3201-4bde-b76f-a845d9bad3f1 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:48.375431 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 167/340] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:19:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:19:48.376940 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c88abdb-de9b-49ba-b563-3c48bcc18a52 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:48.377235 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c88abdb-de9b-49ba-b563-3c48bcc18a52 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:48.377515 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c88abdb-de9b-49ba-b563-3c48bcc18a52 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:48.378956 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 168/341] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:19:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:19:48.383718 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3332d2-8b53-444d-8073-0e6db6cdd248 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:48.383718 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3332d2-8b53-444d-8073-0e6db6cdd248 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:48.383718 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3332d2-8b53-444d-8073-0e6db6cdd248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:48.399711 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-4f3332d2-8b53-444d-8073-0e6db6cdd248 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:19:48.400886 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-4f3332d2-8b53-444d-8073-0e6db6cdd248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:19:48.425616 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74100a8-e7f8-4eb7-8747-8aa9aef08138 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:48.463239 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:48.469753 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca7384d-bf08-4151-a63e-99e8611741c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:48.471222 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca7384d-bf08-4151-a63e-99e8611741c8 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:48.471713 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca7384d-bf08-4151-a63e-99e8611741c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:48.472538 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca7384d-bf08-4151-a63e-99e8611741c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:48.473475 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca7384d-bf08-4151-a63e-99e8611741c8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:19:48.474240 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca7384d-bf08-4151-a63e-99e8611741c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:48.476232 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca7384d-bf08-4151-a63e-99e8611741c8 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:48.490817 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 174/342] 10.222.0.85 () {56 vars in 1101 bytes} [Tue Jun 7 14:19:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:48.492953 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f67e06-4fd4-44b9-8cb2-8405112443e1 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:48.493472 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f67e06-4fd4-44b9-8cb2-8405112443e1 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:48.494648 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f67e06-4fd4-44b9-8cb2-8405112443e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:48.505448 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 169/343] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:19:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 14:19:48.506340 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3332d2-8b53-444d-8073-0e6db6cdd248 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:48.515743 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-d4f67e06-4fd4-44b9-8cb2-8405112443e1 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:19:48.517404 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-d4f67e06-4fd4-44b9-8cb2-8405112443e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:19:48.540951 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:48.621671 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 175/344] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:19:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:19:48.623162 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f67e06-4fd4-44b9-8cb2-8405112443e1 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:48.685401 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3332d2-8b53-444d-8073-0e6db6cdd248 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:48.723872 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ae631e-1f4b-46a7-8e2b-475d51eadeb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:48.725095 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ae631e-1f4b-46a7-8e2b-475d51eadeb1 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:48.725728 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ae631e-1f4b-46a7-8e2b-475d51eadeb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:48.726282 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ae631e-1f4b-46a7-8e2b-475d51eadeb1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:48.748584 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ae631e-1f4b-46a7-8e2b-475d51eadeb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:19:48.757012 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-97ae631e-1f4b-46a7-8e2b-475d51eadeb1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 14:19:48.757012 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:19:48.757012 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:19:48.757012 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:19:48.757012 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:19:48.757012 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:19:48.757012 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:19:48.757012 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:19:48.757012 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:19:48.757012 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:19:48.757012 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:19:48.757012 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:19:48.757012 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:19:48.757012 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:19:48.757012 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:19:48.757012 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:19:48.758032 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:19:48.758032 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:19:48.758032 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:19:48.758032 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:19:48.758032 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:19:48.758032 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:19:48.758032 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:19:48.758032 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:19:48.758032 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:19:48.758032 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:19:48.758032 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:19:48.758032 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:19:48.758032 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:19:48.758032 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:19:48.758032 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:19:48.758032 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:19:48.758819 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:19:48.758819 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:19:48.758819 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:19:48.758819 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:19:48.758819 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:19:48.758819 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:19:48.758819 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:19:48.758819 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:19:48.758819 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:19:48.758819 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:19:48.758819 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 14:19:48.758819 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:19:48.761596 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 170/345] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jun 7 14:19:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:19:48.761885 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f67e06-4fd4-44b9-8cb2-8405112443e1 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:48.762912 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ae631e-1f4b-46a7-8e2b-475d51eadeb1 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:48.799667 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:48.802521 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c47af52-329a-4847-86f6-0374723b8154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:48.803189 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c47af52-329a-4847-86f6-0374723b8154 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:48.803474 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c47af52-329a-4847-86f6-0374723b8154 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:48.803764 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c47af52-329a-4847-86f6-0374723b8154 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:48.825678 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:48.880412 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ae631e-1f4b-46a7-8e2b-475d51eadeb1 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:48.888669 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c47af52-329a-4847-86f6-0374723b8154 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:48.890338 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c47af52-329a-4847-86f6-0374723b8154 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:48.919169 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a620369-16e8-4864-92b0-b897590e232d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:48.920356 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a620369-16e8-4864-92b0-b897590e232d None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:48.920873 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a620369-16e8-4864-92b0-b897590e232d None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:48.921349 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a620369-16e8-4864-92b0-b897590e232d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:48.935156 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a620369-16e8-4864-92b0-b897590e232d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:19:48.944162 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-5a620369-16e8-4864-92b0-b897590e232d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 14:19:48.944162 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:19:48.944162 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:19:48.944162 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:19:48.944162 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:19:48.944162 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:19:48.944162 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:19:48.944162 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:19:48.944162 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:19:48.944162 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:19:48.944162 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:19:48.944162 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:19:48.944162 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:19:48.944162 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:19:48.944162 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:19:48.944162 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:19:48.945120 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:19:48.945120 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:19:48.945120 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:19:48.945120 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:19:48.945120 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:19:48.945120 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:19:48.945120 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:19:48.945120 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:19:48.945120 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:19:48.945120 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:19:48.945120 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:19:48.945120 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:19:48.945120 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:19:48.945120 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:19:48.945120 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:19:48.945120 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:19:48.946067 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:19:48.946067 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:19:48.946067 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:19:48.946067 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:19:48.946067 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:19:48.946067 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:19:48.946067 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:19:48.946067 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:19:48.946067 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:19:48.946067 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:19:48.946067 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 14:19:48.946067 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:19:48.952504 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 171/346] 10.222.0.85 () {56 vars in 1114 bytes} [Tue Jun 7 14:19:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:19:48.953667 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a620369-16e8-4864-92b0-b897590e232d None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:49.024182 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 176/347] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jun 7 14:19:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 14:19:49.025309 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c47af52-329a-4847-86f6-0374723b8154 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:49.058017 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:49.068928 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a620369-16e8-4864-92b0-b897590e232d None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:49.082976 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:49.108564 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f49c21b-b587-44af-9101-76a40b4dea1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:49.109833 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f49c21b-b587-44af-9101-76a40b4dea1b None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:49.110183 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f49c21b-b587-44af-9101-76a40b4dea1b None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:49.110525 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f49c21b-b587-44af-9101-76a40b4dea1b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:49.123393 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f49c21b-b587-44af-9101-76a40b4dea1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:19:49.132665 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-8f49c21b-b587-44af-9101-76a40b4dea1b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 14:19:49.132665 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:19:49.132665 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:19:49.132665 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:19:49.132665 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:19:49.132665 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:19:49.132665 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:19:49.132665 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:19:49.132665 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:19:49.132665 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:19:49.132665 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:19:49.132665 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:19:49.132665 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:19:49.132665 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:19:49.132665 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:19:49.132665 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:19:49.133796 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:19:49.133796 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:19:49.133796 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:19:49.133796 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:19:49.133796 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:19:49.133796 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:19:49.133796 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:19:49.133796 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:19:49.133796 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:19:49.133796 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:19:49.133796 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:19:49.133796 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:19:49.133796 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:19:49.133796 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:19:49.133796 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:19:49.133796 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:19:49.134788 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:19:49.134788 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:19:49.134788 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:19:49.134788 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:19:49.134788 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:19:49.134788 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:19:49.134788 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:19:49.134788 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:19:49.134788 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:19:49.134788 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:19:49.134788 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 14:19:49.134788 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:19:49.136150 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 172/348] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jun 7 14:19:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:19:49.140506 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f49c21b-b587-44af-9101-76a40b4dea1b None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:49.147425 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c47af52-329a-4847-86f6-0374723b8154 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:49.187283 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16ccdf3-e907-41c0-ab79-c29f4ec81952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:49.188630 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16ccdf3-e907-41c0-ab79-c29f4ec81952 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:49.189111 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16ccdf3-e907-41c0-ab79-c29f4ec81952 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:49.189570 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16ccdf3-e907-41c0-ab79-c29f4ec81952 None admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:49.190630 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16ccdf3-e907-41c0-ab79-c29f4ec81952 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:19:49.191138 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16ccdf3-e907-41c0-ab79-c29f4ec81952 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:49.192926 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16ccdf3-e907-41c0-ab79-c29f4ec81952 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:49.203216 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 177/349] 10.222.0.85 () {56 vars in 1101 bytes} [Tue Jun 7 14:19:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:49.205336 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16ccdf3-e907-41c0-ab79-c29f4ec81952 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:49.258915 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f49c21b-b587-44af-9101-76a40b4dea1b None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:49.306156 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473e5b2f-d381-4551-bb95-fe3dc5805794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:49.307132 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473e5b2f-d381-4551-bb95-fe3dc5805794 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:49.307416 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473e5b2f-d381-4551-bb95-fe3dc5805794 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:49.307692 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473e5b2f-d381-4551-bb95-fe3dc5805794 None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:49.309679 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473e5b2f-d381-4551-bb95-fe3dc5805794 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:19:49.309679 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473e5b2f-d381-4551-bb95-fe3dc5805794 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:49.311620 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473e5b2f-d381-4551-bb95-fe3dc5805794 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:49.321378 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 173/350] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jun 7 14:19:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:49.322531 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473e5b2f-d381-4551-bb95-fe3dc5805794 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:49.329459 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16ccdf3-e907-41c0-ab79-c29f4ec81952 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:49.344882 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:49.350437 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:49.375861 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea21982f-30ca-4c3f-aab8-765fec78b314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:49.377034 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea21982f-30ca-4c3f-aab8-765fec78b314 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:49.377634 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea21982f-30ca-4c3f-aab8-765fec78b314 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:49.378109 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea21982f-30ca-4c3f-aab8-765fec78b314 None admin] PATH_INFO: `/v3/projects/f95de357ccad493984042656b2c0689a` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:49.379650 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea21982f-30ca-4c3f-aab8-765fec78b314 None admin] RBAC: Authorizing `identity:get_project(project_id=f95de357ccad493984042656b2c0689a)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:49.381158 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea21982f-30ca-4c3f-aab8-765fec78b314 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:49.383448 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 178/351] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jun 7 14:19:49 2022] GET /identity/v3/projects/f95de357ccad493984042656b2c0689a => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:49.384219 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea21982f-30ca-4c3f-aab8-765fec78b314 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:49.459191 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473e5b2f-d381-4551-bb95-fe3dc5805794 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:49.512349 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df74d74-89cc-4f5d-bffc-40b6b73be733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:49.512349 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df74d74-89cc-4f5d-bffc-40b6b73be733 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:49.512349 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df74d74-89cc-4f5d-bffc-40b6b73be733 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:49.512349 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df74d74-89cc-4f5d-bffc-40b6b73be733 None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:49.512993 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df74d74-89cc-4f5d-bffc-40b6b73be733 None admin] RBAC: Adding query filter params (name=member) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:19:49.512993 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df74d74-89cc-4f5d-bffc-40b6b73be733 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:49.514564 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df74d74-89cc-4f5d-bffc-40b6b73be733 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:49.528560 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 174/352] 10.222.0.85 () {56 vars in 1101 bytes} [Tue Jun 7 14:19:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:49.529718 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c40ad6a-f7cd-434c-887f-efb38acaea50 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:49.530074 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c40ad6a-f7cd-434c-887f-efb38acaea50 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:49.530358 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c40ad6a-f7cd-434c-887f-efb38acaea50 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:49.531865 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 175/353] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:19:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:19:49.532307 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c40ad6a-f7cd-434c-887f-efb38acaea50 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:49.534518 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea21982f-30ca-4c3f-aab8-765fec78b314 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:49.580787 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b87399a-a9ac-42cb-a8c4-e105df7a3224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:49.581501 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b87399a-a9ac-42cb-a8c4-e105df7a3224 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:49.581882 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b87399a-a9ac-42cb-a8c4-e105df7a3224 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:49.582135 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b87399a-a9ac-42cb-a8c4-e105df7a3224 None admin] PATH_INFO: `/v3/users/6093a023afe9438ca266366fb816c855` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:49.583417 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b87399a-a9ac-42cb-a8c4-e105df7a3224 None admin] RBAC: Authorizing `identity:get_user(user_id=6093a023afe9438ca266366fb816c855)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:49.585056 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b87399a-a9ac-42cb-a8c4-e105df7a3224 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:49.586874 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 179/354] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jun 7 14:19:49 2022] GET /identity/v3/users/6093a023afe9438ca266366fb816c855 => generated 293 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:49.589421 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b87399a-a9ac-42cb-a8c4-e105df7a3224 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:49.657547 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:49.667450 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:49.697061 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c40ad6a-f7cd-434c-887f-efb38acaea50 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:49.731039 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b87399a-a9ac-42cb-a8c4-e105df7a3224 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:49.748846 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980f8848-c8b2-499b-8ff8-339516422be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:49.749519 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f8848-c8b2-499b-8ff8-339516422be9 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:49.749813 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f8848-c8b2-499b-8ff8-339516422be9 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:49.750103 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f8848-c8b2-499b-8ff8-339516422be9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:49.775968 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbec692-abc8-49b7-9bd6-89d16f05deb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:49.777346 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbec692-abc8-49b7-9bd6-89d16f05deb0 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:49.777346 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbec692-abc8-49b7-9bd6-89d16f05deb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:49.777547 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbec692-abc8-49b7-9bd6-89d16f05deb0 None admin] PATH_INFO: `/v3/groups/ddcbeccb8288428d8c03e142e4970595` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:49.791961 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbec692-abc8-49b7-9bd6-89d16f05deb0 None admin] RBAC: Authorizing `identity:get_group(group_id=ddcbeccb8288428d8c03e142e4970595)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:49.793590 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbec692-abc8-49b7-9bd6-89d16f05deb0 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:49.796016 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 180/355] 10.222.0.85 () {56 vars in 1180 bytes} [Tue Jun 7 14:19:49 2022] GET /identity/v3/groups/ddcbeccb8288428d8c03e142e4970595 => generated 226 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:49.796884 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbec692-abc8-49b7-9bd6-89d16f05deb0 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:49.854834 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980f8848-c8b2-499b-8ff8-339516422be9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:49.857377 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980f8848-c8b2-499b-8ff8-339516422be9 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:49.933816 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbec692-abc8-49b7-9bd6-89d16f05deb0 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:49.950907 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:49.969253 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:49.986723 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2547c2fd-1e99-468d-98c6-a920884709de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:49.987317 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2547c2fd-1e99-468d-98c6-a920884709de None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:49.987617 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2547c2fd-1e99-468d-98c6-a920884709de None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:49.987904 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2547c2fd-1e99-468d-98c6-a920884709de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:50.000267 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 176/356] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jun 7 14:19:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 14:19:50.001633 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92557ebd-8e48-4786-be89-1ce686eb173c None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:50.001925 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92557ebd-8e48-4786-be89-1ce686eb173c None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:50.002203 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92557ebd-8e48-4786-be89-1ce686eb173c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:50.021929 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-92557ebd-8e48-4786-be89-1ce686eb173c None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:19:50.023956 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-92557ebd-8e48-4786-be89-1ce686eb173c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:19:50.082181 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2547c2fd-1e99-468d-98c6-a920884709de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:50.084034 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2547c2fd-1e99-468d-98c6-a920884709de None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:50.103842 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 177/357] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:19:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:19:50.104954 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92557ebd-8e48-4786-be89-1ce686eb173c None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:50.213343 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 181/358] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jun 7 14:19:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 14:19:50.214277 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2547c2fd-1e99-468d-98c6-a920884709de None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:50.229556 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:50.242335 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92557ebd-8e48-4786-be89-1ce686eb173c None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:50.250497 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:50.288231 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1764676e-fc7c-42ab-82db-f1a037346b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:50.288958 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1764676e-fc7c-42ab-82db-f1a037346b10 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:50.289235 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1764676e-fc7c-42ab-82db-f1a037346b10 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:50.289517 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1764676e-fc7c-42ab-82db-f1a037346b10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:50.290365 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1764676e-fc7c-42ab-82db-f1a037346b10 None admin] RBAC: Adding query filter params (role.id=e2072a2a35d64eec9d32dac7c47c4419, user.id=6093a023afe9438ca266366fb816c855, scope.system=all) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:19:50.290685 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1764676e-fc7c-42ab-82db-f1a037346b10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:50.292133 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1764676e-fc7c-42ab-82db-f1a037346b10 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:50.307506 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 178/359] 10.222.0.85 () {56 vars in 1308 bytes} [Tue Jun 7 14:19:50 2022] GET /identity/v3/role_assignments?user.id=6093a023afe9438ca266366fb816c855&scope.system=all&role.id=e2072a2a35d64eec9d32dac7c47c4419 => generated 510 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:50.309089 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168a6d87-fe44-4572-8fe6-f628bf37466c None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:50.309532 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168a6d87-fe44-4572-8fe6-f628bf37466c None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:50.310078 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168a6d87-fe44-4572-8fe6-f628bf37466c None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:50.311463 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 179/360] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:19:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:19:50.312241 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168a6d87-fe44-4572-8fe6-f628bf37466c None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:50.358890 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2547c2fd-1e99-468d-98c6-a920884709de None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:50.409934 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cde619-c337-498d-9e66-3133109f3ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:50.410899 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cde619-c337-498d-9e66-3133109f3ca5 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:50.411180 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cde619-c337-498d-9e66-3133109f3ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:50.411603 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cde619-c337-498d-9e66-3133109f3ca5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:50.412716 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cde619-c337-498d-9e66-3133109f3ca5 None admin] RBAC: Adding query filter params (group.id=ddcbeccb8288428d8c03e142e4970595, role.id=3771341b3be641139e752aab690397a4, scope.project.id=f95de357ccad493984042656b2c0689a) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:19:50.413023 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cde619-c337-498d-9e66-3133109f3ca5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:50.414705 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cde619-c337-498d-9e66-3133109f3ca5 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:50.425747 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 182/361] 10.222.0.85 () {56 vars in 1376 bytes} [Tue Jun 7 14:19:50 2022] GET /identity/v3/role_assignments?group.id=ddcbeccb8288428d8c03e142e4970595&scope.project.id=f95de357ccad493984042656b2c0689a&role.id=3771341b3be641139e752aab690397a4 => generated 260 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:50.428435 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cde619-c337-498d-9e66-3133109f3ca5 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:50.488039 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168a6d87-fe44-4572-8fe6-f628bf37466c None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:50.550625 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b097f3-fff4-4121-984a-750c4e45a313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:50.552581 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b097f3-fff4-4121-984a-750c4e45a313 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:50.553365 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b097f3-fff4-4121-984a-750c4e45a313 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:50.553786 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b097f3-fff4-4121-984a-750c4e45a313 None admin] PATH_INFO: `/v3/projects/e04d4501005f4310819c6d8307f3df10` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:50.556627 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b097f3-fff4-4121-984a-750c4e45a313 None admin] RBAC: Authorizing `identity:get_project(project_id=e04d4501005f4310819c6d8307f3df10)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:50.558221 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b097f3-fff4-4121-984a-750c4e45a313 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:50.560608 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 180/362] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jun 7 14:19:50 2022] GET /identity/v3/projects/e04d4501005f4310819c6d8307f3df10 => generated 298 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:50.561520 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:50.562283 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b097f3-fff4-4121-984a-750c4e45a313 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:50.594212 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cde619-c337-498d-9e66-3133109f3ca5 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:50.613679 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:50.656676 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb73b1e8-9160-4f4d-bdbd-7b86950474e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:50.657531 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb73b1e8-9160-4f4d-bdbd-7b86950474e0 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:50.657820 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb73b1e8-9160-4f4d-bdbd-7b86950474e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:50.658102 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb73b1e8-9160-4f4d-bdbd-7b86950474e0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:50.683233 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb73b1e8-9160-4f4d-bdbd-7b86950474e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:19:50.692400 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-eb73b1e8-9160-4f4d-bdbd-7b86950474e0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 14:19:50.692400 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:19:50.692400 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:19:50.692400 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:19:50.692400 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:19:50.692400 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:19:50.692400 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:19:50.692400 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:19:50.692400 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:19:50.692400 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:19:50.692400 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:19:50.692400 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:19:50.692400 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:19:50.692400 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:19:50.692400 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:19:50.692400 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:19:50.693339 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:19:50.693339 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:19:50.693339 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:19:50.693339 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:19:50.693339 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:19:50.693339 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:19:50.693339 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:19:50.693339 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:19:50.693339 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:19:50.693339 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:19:50.693339 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:19:50.693339 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:19:50.693339 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:19:50.693339 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:19:50.693339 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:19:50.693339 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:19:50.694132 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:19:50.694132 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:19:50.694132 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:19:50.694132 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:19:50.694132 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:19:50.694132 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:19:50.694132 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:19:50.694132 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:19:50.694132 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:19:50.694132 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:19:50.694132 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 14:19:50.694132 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:19:50.694731 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 183/363] 10.222.0.85 () {56 vars in 1114 bytes} [Tue Jun 7 14:19:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:19:50.699003 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4402c67a-6af4-4cae-a236-f3b10af3bd2e None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:50.699003 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4402c67a-6af4-4cae-a236-f3b10af3bd2e None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:50.699003 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4402c67a-6af4-4cae-a236-f3b10af3bd2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:50.712721 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b097f3-fff4-4121-984a-750c4e45a313 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:50.718912 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-4402c67a-6af4-4cae-a236-f3b10af3bd2e None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:19:50.720154 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-4402c67a-6af4-4cae-a236-f3b10af3bd2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:19:50.766443 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52792ff9-15f5-48bb-98b6-8a8bcc98569e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:50.767366 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52792ff9-15f5-48bb-98b6-8a8bcc98569e None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:50.767658 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52792ff9-15f5-48bb-98b6-8a8bcc98569e None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:50.767930 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52792ff9-15f5-48bb-98b6-8a8bcc98569e None admin] PATH_INFO: `/v3/projects/e04d4501005f4310819c6d8307f3df10` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:50.770469 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52792ff9-15f5-48bb-98b6-8a8bcc98569e None admin] RBAC: Authorizing `identity:get_project(project_id=e04d4501005f4310819c6d8307f3df10)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:50.771952 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52792ff9-15f5-48bb-98b6-8a8bcc98569e None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:50.774975 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 181/364] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jun 7 14:19:50 2022] GET /identity/v3/projects/e04d4501005f4310819c6d8307f3df10 => generated 298 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:50.778550 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52792ff9-15f5-48bb-98b6-8a8bcc98569e None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:50.826797 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 184/365] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:19:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:19:50.828623 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4402c67a-6af4-4cae-a236-f3b10af3bd2e None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:50.927566 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:50.929787 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:50.937377 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52792ff9-15f5-48bb-98b6-8a8bcc98569e None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:50.985005 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33d50a4-7d5b-440f-91fc-781423cf1c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:50.985878 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33d50a4-7d5b-440f-91fc-781423cf1c7b None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:50.986357 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33d50a4-7d5b-440f-91fc-781423cf1c7b None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:50.986633 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33d50a4-7d5b-440f-91fc-781423cf1c7b None admin] PATH_INFO: `/v3/groups/ddcbeccb8288428d8c03e142e4970595` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:50.987853 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33d50a4-7d5b-440f-91fc-781423cf1c7b None admin] RBAC: Authorizing `identity:get_group(group_id=ddcbeccb8288428d8c03e142e4970595)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:50.989349 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33d50a4-7d5b-440f-91fc-781423cf1c7b None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:50.991197 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 182/366] 10.222.0.85 () {56 vars in 1180 bytes} [Tue Jun 7 14:19:50 2022] GET /identity/v3/groups/ddcbeccb8288428d8c03e142e4970595 => generated 226 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:50.993997 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33d50a4-7d5b-440f-91fc-781423cf1c7b None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:51.001699 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4402c67a-6af4-4cae-a236-f3b10af3bd2e None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:51.066159 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa45a96c-f860-4266-ab13-1c5698406924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:51.066879 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa45a96c-f860-4266-ab13-1c5698406924 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:51.067166 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa45a96c-f860-4266-ab13-1c5698406924 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:51.067442 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa45a96c-f860-4266-ab13-1c5698406924 None admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:51.068204 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa45a96c-f860-4266-ab13-1c5698406924 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:19:51.068730 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa45a96c-f860-4266-ab13-1c5698406924 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:51.070769 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa45a96c-f860-4266-ab13-1c5698406924 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:51.080292 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 185/367] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jun 7 14:19:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:51.082645 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa45a96c-f860-4266-ab13-1c5698406924 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:51.137479 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33d50a4-7d5b-440f-91fc-781423cf1c7b None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:51.189424 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5ebd3a-041b-450a-81d7-94482090f0aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:51.190299 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5ebd3a-041b-450a-81d7-94482090f0aa None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:51.190586 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5ebd3a-041b-450a-81d7-94482090f0aa None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:51.190859 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5ebd3a-041b-450a-81d7-94482090f0aa None admin] PATH_INFO: `/v3/groups/ddcbeccb8288428d8c03e142e4970595` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:51.192648 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5ebd3a-041b-450a-81d7-94482090f0aa None admin] RBAC: Authorizing `identity:get_group(group_id=ddcbeccb8288428d8c03e142e4970595)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:51.194486 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5ebd3a-041b-450a-81d7-94482090f0aa None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:51.197386 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 183/368] 10.222.0.85 () {56 vars in 1180 bytes} [Tue Jun 7 14:19:50 2022] GET /identity/v3/groups/ddcbeccb8288428d8c03e142e4970595 => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:51.198726 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5ebd3a-041b-450a-81d7-94482090f0aa None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:51.209303 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa45a96c-f860-4266-ab13-1c5698406924 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:51.249426 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:51.250054 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:51.258987 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970514b1-8dcd-4ad9-a02e-41a108875fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:51.260445 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970514b1-8dcd-4ad9-a02e-41a108875fa3 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:51.260903 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970514b1-8dcd-4ad9-a02e-41a108875fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:51.264554 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970514b1-8dcd-4ad9-a02e-41a108875fa3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:51.295045 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970514b1-8dcd-4ad9-a02e-41a108875fa3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:19:51.305815 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-970514b1-8dcd-4ad9-a02e-41a108875fa3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:19:51.305815 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:19:51.305815 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:19:51.305815 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:19:51.305815 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:19:51.305815 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:19:51.305815 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:19:51.305815 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:19:51.305815 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:19:51.305815 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:19:51.305815 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:19:51.305815 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:19:51.305815 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:19:51.305815 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:19:51.305815 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:19:51.305815 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:19:51.307042 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:19:51.307042 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:19:51.307042 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:19:51.307042 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:19:51.307042 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:19:51.307042 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:19:51.307042 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:19:51.307042 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:19:51.307042 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:19:51.307042 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:19:51.307042 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:19:51.307042 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:19:51.307042 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:19:51.307042 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:19:51.307042 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:19:51.307042 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:19:51.307812 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:19:51.307812 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:19:51.307812 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:19:51.307812 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:19:51.307812 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:19:51.307812 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:19:51.307812 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:19:51.307812 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:19:51.307812 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:19:51.307812 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:19:51.307812 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:19:51.307812 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:19:51.312487 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 186/369] 10.222.0.85 () {56 vars in 1096 bytes} [Tue Jun 7 14:19:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:19:51.313435 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970514b1-8dcd-4ad9-a02e-41a108875fa3 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:51.339870 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5ebd3a-041b-450a-81d7-94482090f0aa None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:51.396055 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e3c0f3-0bf4-48b3-817d-0c3e9f0f72a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:51.396838 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e3c0f3-0bf4-48b3-817d-0c3e9f0f72a3 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:51.397133 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e3c0f3-0bf4-48b3-817d-0c3e9f0f72a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:51.397414 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e3c0f3-0bf4-48b3-817d-0c3e9f0f72a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:51.398830 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e3c0f3-0bf4-48b3-817d-0c3e9f0f72a3 None admin] RBAC: Adding query filter params (group.id=ddcbeccb8288428d8c03e142e4970595, role.id=494cd000d6884a179c334848f52a3a21, scope.project.id=e04d4501005f4310819c6d8307f3df10) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:19:51.399154 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e3c0f3-0bf4-48b3-817d-0c3e9f0f72a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:51.400652 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e3c0f3-0bf4-48b3-817d-0c3e9f0f72a3 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:51.406582 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 184/370] 10.222.0.85 () {56 vars in 1376 bytes} [Tue Jun 7 14:19:51 2022] GET /identity/v3/role_assignments?group.id=ddcbeccb8288428d8c03e142e4970595&scope.project.id=e04d4501005f4310819c6d8307f3df10&role.id=494cd000d6884a179c334848f52a3a21 => generated 260 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:51.407738 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e3c0f3-0bf4-48b3-817d-0c3e9f0f72a3 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:51.457589 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970514b1-8dcd-4ad9-a02e-41a108875fa3 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:51.502022 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e1ba0e-4aae-475d-ae4b-9c0b6fc65979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:51.502972 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e1ba0e-4aae-475d-ae4b-9c0b6fc65979 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:51.503422 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e1ba0e-4aae-475d-ae4b-9c0b6fc65979 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:51.503892 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e1ba0e-4aae-475d-ae4b-9c0b6fc65979 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:51.521537 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:51.531706 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e3c0f3-0bf4-48b3-817d-0c3e9f0f72a3 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:51.585261 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707ff0ba-3ecf-4fc6-8981-ab0e5317507d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:51.588483 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707ff0ba-3ecf-4fc6-8981-ab0e5317507d None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:51.589028 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707ff0ba-3ecf-4fc6-8981-ab0e5317507d None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:51.589500 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707ff0ba-3ecf-4fc6-8981-ab0e5317507d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:51.590629 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707ff0ba-3ecf-4fc6-8981-ab0e5317507d None admin] RBAC: Adding query filter params (group.id=ddcbeccb8288428d8c03e142e4970595, role.id=3771341b3be641139e752aab690397a4, scope.project.id=e04d4501005f4310819c6d8307f3df10) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:19:51.591267 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707ff0ba-3ecf-4fc6-8981-ab0e5317507d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:51.593242 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707ff0ba-3ecf-4fc6-8981-ab0e5317507d None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:51.600469 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 185/371] 10.222.0.85 () {56 vars in 1376 bytes} [Tue Jun 7 14:19:51 2022] GET /identity/v3/role_assignments?group.id=ddcbeccb8288428d8c03e142e4970595&scope.project.id=e04d4501005f4310819c6d8307f3df10&role.id=3771341b3be641139e752aab690397a4 => generated 260 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:51.601850 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707ff0ba-3ecf-4fc6-8981-ab0e5317507d None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:51.623116 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:51.657381 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e1ba0e-4aae-475d-ae4b-9c0b6fc65979 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:51.659512 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e1ba0e-4aae-475d-ae4b-9c0b6fc65979 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:51.835159 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 187/372] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jun 7 14:19:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 14:19:51.845397 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e1ba0e-4aae-475d-ae4b-9c0b6fc65979 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:51.857770 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707ff0ba-3ecf-4fc6-8981-ab0e5317507d None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:51.934089 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c88446e-8d53-4111-b4db-f9a1def8ccbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:51.934787 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c88446e-8d53-4111-b4db-f9a1def8ccbe None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:51.935077 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c88446e-8d53-4111-b4db-f9a1def8ccbe None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:51.935355 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c88446e-8d53-4111-b4db-f9a1def8ccbe None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:51.936128 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c88446e-8d53-4111-b4db-f9a1def8ccbe None admin] RBAC: Adding query filter params (name=admin) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:19:51.936911 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c88446e-8d53-4111-b4db-f9a1def8ccbe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:51.938330 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c88446e-8d53-4111-b4db-f9a1def8ccbe None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:51.972168 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 186/373] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jun 7 14:19:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:51.981391 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c88446e-8d53-4111-b4db-f9a1def8ccbe None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:51.982615 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:52.012648 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e1ba0e-4aae-475d-ae4b-9c0b6fc65979 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:52.014178 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:52.107364 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c88446e-8d53-4111-b4db-f9a1def8ccbe None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:52.109140 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322bee41-1c3a-4fe4-a3e2-fec173e41837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:52.110036 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322bee41-1c3a-4fe4-a3e2-fec173e41837 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:52.110327 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322bee41-1c3a-4fe4-a3e2-fec173e41837 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:52.110611 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322bee41-1c3a-4fe4-a3e2-fec173e41837 None admin] PATH_INFO: `/v3/projects/f95de357ccad493984042656b2c0689a` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:52.111926 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322bee41-1c3a-4fe4-a3e2-fec173e41837 None admin] RBAC: Authorizing `identity:get_project(project_id=f95de357ccad493984042656b2c0689a)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:52.113422 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322bee41-1c3a-4fe4-a3e2-fec173e41837 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:52.121937 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 188/374] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jun 7 14:19:51 2022] GET /identity/v3/projects/f95de357ccad493984042656b2c0689a => generated 302 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:19:52.124132 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322bee41-1c3a-4fe4-a3e2-fec173e41837 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:52.180574 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f053c91c-2215-48c8-8ce0-29daf716f2d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:52.181247 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f053c91c-2215-48c8-8ce0-29daf716f2d6 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:52.181531 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f053c91c-2215-48c8-8ce0-29daf716f2d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:52.181809 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f053c91c-2215-48c8-8ce0-29daf716f2d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:52.283525 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f053c91c-2215-48c8-8ce0-29daf716f2d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:52.285488 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f053c91c-2215-48c8-8ce0-29daf716f2d6 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:52.320986 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:52.411647 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322bee41-1c3a-4fe4-a3e2-fec173e41837 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:52.418596 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 187/375] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jun 7 14:19:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 14:19:52.425953 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f053c91c-2215-48c8-8ce0-29daf716f2d6 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:52.548287 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f053c91c-2215-48c8-8ce0-29daf716f2d6 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:52.559790 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ce444b-55f2-4c50-a359-c148c78fa3f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:52.561261 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ce444b-55f2-4c50-a359-c148c78fa3f7 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:52.561764 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ce444b-55f2-4c50-a359-c148c78fa3f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:52.562262 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ce444b-55f2-4c50-a359-c148c78fa3f7 None admin] PATH_INFO: `/v3/groups/ddcbeccb8288428d8c03e142e4970595` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:52.569687 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ce444b-55f2-4c50-a359-c148c78fa3f7 None admin] RBAC: Authorizing `identity:get_group(group_id=ddcbeccb8288428d8c03e142e4970595)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:52.572773 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ce444b-55f2-4c50-a359-c148c78fa3f7 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:52.582854 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 189/376] 10.222.0.85 () {56 vars in 1180 bytes} [Tue Jun 7 14:19:52 2022] GET /identity/v3/groups/ddcbeccb8288428d8c03e142e4970595 => generated 226 bytes in 459 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:52.583785 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:52.588843 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ce444b-55f2-4c50-a359-c148c78fa3f7 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:52.590761 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab502ae7-1239-41bd-a282-f1c3d74027fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:52.591647 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab502ae7-1239-41bd-a282-f1c3d74027fd None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:52.603754 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab502ae7-1239-41bd-a282-f1c3d74027fd None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:52.603754 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab502ae7-1239-41bd-a282-f1c3d74027fd None admin] PATH_INFO: `/v3/projects/61703df7c02543d6ba1f949d9811ee43` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:52.603754 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab502ae7-1239-41bd-a282-f1c3d74027fd None admin] RBAC: Authorizing `identity:get_project(project_id=61703df7c02543d6ba1f949d9811ee43)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:52.603754 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab502ae7-1239-41bd-a282-f1c3d74027fd None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:52.603754 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 188/377] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jun 7 14:19:52 2022] GET /identity/v3/projects/61703df7c02543d6ba1f949d9811ee43 => generated 344 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:52.605665 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab502ae7-1239-41bd-a282-f1c3d74027fd None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:52.659412 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:52.795512 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ce444b-55f2-4c50-a359-c148c78fa3f7 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:52.842657 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fc6e1f-398b-43eb-ad99-bcbc670cc3e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:52.843335 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc6e1f-398b-43eb-ad99-bcbc670cc3e9 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:52.843625 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc6e1f-398b-43eb-ad99-bcbc670cc3e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:52.843904 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc6e1f-398b-43eb-ad99-bcbc670cc3e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:52.844799 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fc6e1f-398b-43eb-ad99-bcbc670cc3e9 None admin] RBAC: Adding query filter params (group.id=ddcbeccb8288428d8c03e142e4970595, role.id=494cd000d6884a179c334848f52a3a21, scope.project.id=f95de357ccad493984042656b2c0689a) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:19:52.845117 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fc6e1f-398b-43eb-ad99-bcbc670cc3e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:52.847317 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fc6e1f-398b-43eb-ad99-bcbc670cc3e9 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:52.853490 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 190/378] 10.222.0.85 () {56 vars in 1376 bytes} [Tue Jun 7 14:19:52 2022] GET /identity/v3/role_assignments?group.id=ddcbeccb8288428d8c03e142e4970595&scope.project.id=f95de357ccad493984042656b2c0689a&role.id=494cd000d6884a179c334848f52a3a21 => generated 260 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:52.916104 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab502ae7-1239-41bd-a282-f1c3d74027fd None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:52.990667 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe3af32-52b7-48ca-9955-19562095dd7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:52.991567 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe3af32-52b7-48ca-9955-19562095dd7a None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:52.991855 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe3af32-52b7-48ca-9955-19562095dd7a None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:52.992747 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe3af32-52b7-48ca-9955-19562095dd7a None admin] PATH_INFO: `/v3/groups/c1f2253e8d3a4fff9f5f3e3bbc9a247f` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:53.003533 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe3af32-52b7-48ca-9955-19562095dd7a None admin] RBAC: Authorizing `identity:get_group(group_id=c1f2253e8d3a4fff9f5f3e3bbc9a247f)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:53.005066 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe3af32-52b7-48ca-9955-19562095dd7a None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:53.008217 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 189/379] 10.222.0.85 () {56 vars in 1180 bytes} [Tue Jun 7 14:19:52 2022] GET /identity/v3/groups/c1f2253e8d3a4fff9f5f3e3bbc9a247f => generated 229 bytes in 402 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:53.013369 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fc6e1f-398b-43eb-ad99-bcbc670cc3e9 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:53.138508 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:53.148634 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fc6e1f-398b-43eb-ad99-bcbc670cc3e9 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:53.196213 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f6ad40-50d4-413f-ab1d-a079e99e3175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:53.196955 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f6ad40-50d4-413f-ab1d-a079e99e3175 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:53.197895 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f6ad40-50d4-413f-ab1d-a079e99e3175 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:53.198302 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f6ad40-50d4-413f-ab1d-a079e99e3175 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:53.199260 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f6ad40-50d4-413f-ab1d-a079e99e3175 None admin] RBAC: Adding query filter params (group.id=c1f2253e8d3a4fff9f5f3e3bbc9a247f, role.id=9baff2b18f2649f7b757ff2e1f63c853, scope.project.id=61703df7c02543d6ba1f949d9811ee43) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:19:53.199583 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f6ad40-50d4-413f-ab1d-a079e99e3175 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:53.201076 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f6ad40-50d4-413f-ab1d-a079e99e3175 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:53.208044 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 191/380] 10.222.0.85 () {56 vars in 1376 bytes} [Tue Jun 7 14:19:53 2022] GET /identity/v3/role_assignments?group.id=c1f2253e8d3a4fff9f5f3e3bbc9a247f&scope.project.id=61703df7c02543d6ba1f949d9811ee43&role.id=9baff2b18f2649f7b757ff2e1f63c853 => generated 260 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:53.253114 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6030195-1585-4085-80dd-8d880e86d837 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:53.253620 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6030195-1585-4085-80dd-8d880e86d837 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:53.253944 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6030195-1585-4085-80dd-8d880e86d837 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:53.256117 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 190/381] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:19:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:19:53.261748 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b5f371-083f-4cf8-9b59-e88134fb204b None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:53.262051 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b5f371-083f-4cf8-9b59-e88134fb204b None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:53.262314 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b5f371-083f-4cf8-9b59-e88134fb204b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:53.290217 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-a1b5f371-083f-4cf8-9b59-e88134fb204b None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:19:53.291448 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-a1b5f371-083f-4cf8-9b59-e88134fb204b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:19:53.400952 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 192/382] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:19:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 14:19:53.440622 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6030195-1585-4085-80dd-8d880e86d837 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:53.524636 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:53.614032 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6030195-1585-4085-80dd-8d880e86d837 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:53.657301 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9173dc6-85a7-4975-b5c5-7db77f932c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:53.658034 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9173dc6-85a7-4975-b5c5-7db77f932c87 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:53.658321 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9173dc6-85a7-4975-b5c5-7db77f932c87 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:53.658603 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9173dc6-85a7-4975-b5c5-7db77f932c87 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:53.681457 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9173dc6-85a7-4975-b5c5-7db77f932c87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:19:53.689536 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-a9173dc6-85a7-4975-b5c5-7db77f932c87 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 14:19:53.689536 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:19:53.689536 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:19:53.689536 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:19:53.689536 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:19:53.689536 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:19:53.689536 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:19:53.689536 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:19:53.689536 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:19:53.689536 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:19:53.689536 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:19:53.689536 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:19:53.689536 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:19:53.689536 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:19:53.689536 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:19:53.689536 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:19:53.690452 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:19:53.690452 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:19:53.690452 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:19:53.690452 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:19:53.690452 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:19:53.690452 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:19:53.690452 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:19:53.690452 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:19:53.690452 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:19:53.690452 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:19:53.690452 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:19:53.690452 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:19:53.690452 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:19:53.690452 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:19:53.690452 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:19:53.690452 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:19:53.691245 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:19:53.691245 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:19:53.691245 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:19:53.691245 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:19:53.691245 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:19:53.691245 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:19:53.691245 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:19:53.691245 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:19:53.691245 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:19:53.691245 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:19:53.691245 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 14:19:53.691245 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:19:53.691245 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 191/383] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jun 7 14:19:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:19:53.698376 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b5f371-083f-4cf8-9b59-e88134fb204b None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:53.802288 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:53.831337 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b5f371-083f-4cf8-9b59-e88134fb204b None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:53.879590 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bb8713-1923-47c7-93e2-a2f2cc2fe3b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:53.880288 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bb8713-1923-47c7-93e2-a2f2cc2fe3b8 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:53.880601 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bb8713-1923-47c7-93e2-a2f2cc2fe3b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:53.880880 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bb8713-1923-47c7-93e2-a2f2cc2fe3b8 None admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:53.881834 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bb8713-1923-47c7-93e2-a2f2cc2fe3b8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:19:53.882148 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bb8713-1923-47c7-93e2-a2f2cc2fe3b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:53.884360 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bb8713-1923-47c7-93e2-a2f2cc2fe3b8 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:53.899100 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 193/384] 10.222.0.85 () {56 vars in 1101 bytes} [Tue Jun 7 14:19:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:53.900151 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9173dc6-85a7-4975-b5c5-7db77f932c87 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:54.047445 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9173dc6-85a7-4975-b5c5-7db77f932c87 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:54.081627 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:54.105840 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd03ddd-5c99-412d-a862-e51e873e2060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:54.106720 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd03ddd-5c99-412d-a862-e51e873e2060 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:54.107006 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd03ddd-5c99-412d-a862-e51e873e2060 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:54.107284 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd03ddd-5c99-412d-a862-e51e873e2060 None admin] PATH_INFO: `/v3/groups/ddcbeccb8288428d8c03e142e4970595` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:54.108524 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd03ddd-5c99-412d-a862-e51e873e2060 None admin] RBAC: Authorizing `identity:get_group(group_id=ddcbeccb8288428d8c03e142e4970595)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:54.110048 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd03ddd-5c99-412d-a862-e51e873e2060 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:54.112133 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 192/385] 10.222.0.85 () {56 vars in 1180 bytes} [Tue Jun 7 14:19:53 2022] GET /identity/v3/groups/ddcbeccb8288428d8c03e142e4970595 => generated 226 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:19:54.118155 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bb8713-1923-47c7-93e2-a2f2cc2fe3b8 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:54.348461 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bb8713-1923-47c7-93e2-a2f2cc2fe3b8 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:54.424304 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf61b40-0759-4791-8330-a10f50a2f904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:54.426538 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf61b40-0759-4791-8330-a10f50a2f904 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:54.427563 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf61b40-0759-4791-8330-a10f50a2f904 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:54.428410 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf61b40-0759-4791-8330-a10f50a2f904 None admin] PATH_INFO: `/v3/projects/f95de357ccad493984042656b2c0689a` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:54.438031 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf61b40-0759-4791-8330-a10f50a2f904 None admin] RBAC: Authorizing `identity:get_project(project_id=f95de357ccad493984042656b2c0689a)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:54.441637 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf61b40-0759-4791-8330-a10f50a2f904 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:54.446859 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 194/386] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jun 7 14:19:54 2022] GET /identity/v3/projects/f95de357ccad493984042656b2c0689a => generated 302 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:19:54.455473 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd03ddd-5c99-412d-a862-e51e873e2060 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:54.493792 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e873b5b1-512f-49e4-a991-9323720a1c06 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:54.494563 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e873b5b1-512f-49e4-a991-9323720a1c06 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:54.494988 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e873b5b1-512f-49e4-a991-9323720a1c06 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:54.496867 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 195/387] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:19:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:19:54.504784 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c76ab7-e20c-4c46-b073-f8601fc7684b None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:54.505383 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c76ab7-e20c-4c46-b073-f8601fc7684b None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:54.505880 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c76ab7-e20c-4c46-b073-f8601fc7684b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:54.538675 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-c8c76ab7-e20c-4c46-b073-f8601fc7684b None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:19:54.540887 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-c8c76ab7-e20c-4c46-b073-f8601fc7684b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:19:54.549799 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:54.604475 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd03ddd-5c99-412d-a862-e51e873e2060 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:54.614331 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 196/388] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:19:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 14:19:54.621013 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998bd283-9c5d-4e9e-91fa-88e39ef27612 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:54.621374 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998bd283-9c5d-4e9e-91fa-88e39ef27612 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:54.621730 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998bd283-9c5d-4e9e-91fa-88e39ef27612 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:54.623172 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 197/389] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:19:54 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:19:54.628849 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee18d9-d65d-4e01-92e0-c51f41f4474d None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:54.629059 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee18d9-d65d-4e01-92e0-c51f41f4474d None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:54.629398 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee18d9-d65d-4e01-92e0-c51f41f4474d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:54.646314 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef10b40b-9781-4da3-b37f-7e11a1aed6df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:54.647553 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef10b40b-9781-4da3-b37f-7e11a1aed6df None admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:54.648103 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef10b40b-9781-4da3-b37f-7e11a1aed6df None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:54.648732 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef10b40b-9781-4da3-b37f-7e11a1aed6df None admin] PATH_INFO: `/v3/projects/f95de357ccad493984042656b2c0689a/groups/ddcbeccb8288428d8c03e142e4970595/roles/3771341b3be641139e752aab690397a4` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:54.651011 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef10b40b-9781-4da3-b37f-7e11a1aed6df None admin] RBAC: Authorizing `identity:create_grant(project_id=f95de357ccad493984042656b2c0689a, group_id=ddcbeccb8288428d8c03e142e4970595, role_id=3771341b3be641139e752aab690397a4)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:54.654786 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef10b40b-9781-4da3-b37f-7e11a1aed6df None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:54.661643 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-4eee18d9-d65d-4e01-92e0-c51f41f4474d None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:19:54.663115 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-4eee18d9-d65d-4e01-92e0-c51f41f4474d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:19:54.740421 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 198/390] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:19:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:19:54.740421 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eee18d9-d65d-4e01-92e0-c51f41f4474d None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:54.772199 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 193/391] 10.222.0.85 () {58 vars in 1442 bytes} [Tue Jun 7 14:19:54 2022] PUT /identity/v3/projects/f95de357ccad493984042656b2c0689a/groups/ddcbeccb8288428d8c03e142e4970595/roles/3771341b3be641139e752aab690397a4 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:19:54.791753 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef10b40b-9781-4da3-b37f-7e11a1aed6df None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:54.815834 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:54.866858 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eee18d9-d65d-4e01-92e0-c51f41f4474d None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:54.876503 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:54.911029 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6335e3-4b51-41c9-ae59-19edfcf4e598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:54.912030 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6335e3-4b51-41c9-ae59-19edfcf4e598 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:54.913057 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6335e3-4b51-41c9-ae59-19edfcf4e598 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:54.913057 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6335e3-4b51-41c9-ae59-19edfcf4e598 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:54.925326 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6335e3-4b51-41c9-ae59-19edfcf4e598 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:19:54.934090 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-fd6335e3-4b51-41c9-ae59-19edfcf4e598 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 14:19:54.934090 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:19:54.934090 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:19:54.934090 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:19:54.934090 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:19:54.934090 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:19:54.934090 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:19:54.934090 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:19:54.934090 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:19:54.934090 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:19:54.934090 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:19:54.934090 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:19:54.934090 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:19:54.934090 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:19:54.934090 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:19:54.934090 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:19:54.940007 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:19:54.940007 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:19:54.940007 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:19:54.940007 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:19:54.940007 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:19:54.940007 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:19:54.940007 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:19:54.940007 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:19:54.940007 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:19:54.940007 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:19:54.940007 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:19:54.940007 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:19:54.940007 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:19:54.940007 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:19:54.940007 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:19:54.940007 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:19:54.940829 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:19:54.940829 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:19:54.940829 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:19:54.940829 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:19:54.940829 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:19:54.940829 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:19:54.940829 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:19:54.940829 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:19:54.940829 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:19:54.940829 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:19:54.940829 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 14:19:54.940829 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:19:54.940829 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 199/392] 10.222.0.85 () {56 vars in 1114 bytes} [Tue Jun 7 14:19:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:19:54.942198 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6335e3-4b51-41c9-ae59-19edfcf4e598 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:54.983830 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef10b40b-9781-4da3-b37f-7e11a1aed6df None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:55.060066 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acbdaf0-2049-4f8d-8e96-a9187ebbfc4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:55.061477 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acbdaf0-2049-4f8d-8e96-a9187ebbfc4f None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:55.061981 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acbdaf0-2049-4f8d-8e96-a9187ebbfc4f None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:55.062578 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acbdaf0-2049-4f8d-8e96-a9187ebbfc4f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:55.080831 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acbdaf0-2049-4f8d-8e96-a9187ebbfc4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:19:55.095545 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-2acbdaf0-2049-4f8d-8e96-a9187ebbfc4f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 14:19:55.095545 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:19:55.095545 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:19:55.095545 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:19:55.095545 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:19:55.095545 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:19:55.095545 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:19:55.095545 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:19:55.095545 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:19:55.095545 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:19:55.095545 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:19:55.095545 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:19:55.095545 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:19:55.095545 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:19:55.095545 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:19:55.095545 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:19:55.096535 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:19:55.096535 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:19:55.096535 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:19:55.096535 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:19:55.096535 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:19:55.096535 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:19:55.096535 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:19:55.096535 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:19:55.096535 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:19:55.096535 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:19:55.096535 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:19:55.096535 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:19:55.096535 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:19:55.096535 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:19:55.096535 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:19:55.096535 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:19:55.097342 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:19:55.097342 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:19:55.097342 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:19:55.097342 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:19:55.097342 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:19:55.097342 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:19:55.097342 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:19:55.097342 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:19:55.097342 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:19:55.097342 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:19:55.097342 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 14:19:55.097342 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:19:55.097934 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 194/393] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jun 7 14:19:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:19:55.106866 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acbdaf0-2049-4f8d-8e96-a9187ebbfc4f None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:55.197795 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6335e3-4b51-41c9-ae59-19edfcf4e598 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:55.236222 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acbdaf0-2049-4f8d-8e96-a9187ebbfc4f None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:55.242491 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:55.245489 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:55.250664 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6670779a-26c8-4a3b-ac6d-bfe203e8db6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:55.251406 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6670779a-26c8-4a3b-ac6d-bfe203e8db6c None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:55.251694 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6670779a-26c8-4a3b-ac6d-bfe203e8db6c None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:55.251977 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6670779a-26c8-4a3b-ac6d-bfe203e8db6c None admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:55.252781 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6670779a-26c8-4a3b-ac6d-bfe203e8db6c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:19:55.253099 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6670779a-26c8-4a3b-ac6d-bfe203e8db6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:55.254703 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6670779a-26c8-4a3b-ac6d-bfe203e8db6c None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:55.267438 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 200/394] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jun 7 14:19:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:55.274480 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6670779a-26c8-4a3b-ac6d-bfe203e8db6c None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:55.281573 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a790377-cccb-4d83-aa96-edd5eba550af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:55.282556 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a790377-cccb-4d83-aa96-edd5eba550af None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:55.283091 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a790377-cccb-4d83-aa96-edd5eba550af None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:55.283541 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a790377-cccb-4d83-aa96-edd5eba550af None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:55.284996 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a790377-cccb-4d83-aa96-edd5eba550af None admin] RBAC: Adding query filter params (name=member) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:19:55.285523 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a790377-cccb-4d83-aa96-edd5eba550af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:55.287146 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a790377-cccb-4d83-aa96-edd5eba550af None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:55.299329 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 195/395] 10.222.0.85 () {56 vars in 1101 bytes} [Tue Jun 7 14:19:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:55.307563 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a790377-cccb-4d83-aa96-edd5eba550af None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:55.434778 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6670779a-26c8-4a3b-ac6d-bfe203e8db6c None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:55.488772 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a790377-cccb-4d83-aa96-edd5eba550af None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:55.516578 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43570c94-fd37-4990-83b6-f926670d71c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:55.517795 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43570c94-fd37-4990-83b6-f926670d71c4 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:55.518397 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43570c94-fd37-4990-83b6-f926670d71c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:55.518728 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43570c94-fd37-4990-83b6-f926670d71c4 None admin] PATH_INFO: `/v3/groups/ddcbeccb8288428d8c03e142e4970595` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:55.520198 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43570c94-fd37-4990-83b6-f926670d71c4 None admin] RBAC: Authorizing `identity:get_group(group_id=ddcbeccb8288428d8c03e142e4970595)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:55.521917 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43570c94-fd37-4990-83b6-f926670d71c4 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:55.524089 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 201/396] 10.222.0.85 () {56 vars in 1180 bytes} [Tue Jun 7 14:19:55 2022] GET /identity/v3/groups/ddcbeccb8288428d8c03e142e4970595 => generated 226 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:55.536406 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43570c94-fd37-4990-83b6-f926670d71c4 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:55.538338 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9a4e9d-8baf-4f35-ba51-1d698942a11b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:55.543837 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9a4e9d-8baf-4f35-ba51-1d698942a11b None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:55.544121 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9a4e9d-8baf-4f35-ba51-1d698942a11b None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:55.544416 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9a4e9d-8baf-4f35-ba51-1d698942a11b None admin] PATH_INFO: `/v3/groups/ddcbeccb8288428d8c03e142e4970595` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:55.547599 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9a4e9d-8baf-4f35-ba51-1d698942a11b None admin] RBAC: Authorizing `identity:get_group(group_id=ddcbeccb8288428d8c03e142e4970595)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:55.549090 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9a4e9d-8baf-4f35-ba51-1d698942a11b None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:55.552435 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 196/397] 10.222.0.85 () {56 vars in 1180 bytes} [Tue Jun 7 14:19:55 2022] GET /identity/v3/groups/ddcbeccb8288428d8c03e142e4970595 => generated 226 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:55.557330 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9a4e9d-8baf-4f35-ba51-1d698942a11b None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:55.651098 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:55.666428 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:55.731254 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43570c94-fd37-4990-83b6-f926670d71c4 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:55.798897 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9a4e9d-8baf-4f35-ba51-1d698942a11b None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:55.808150 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608e6a26-80c1-44e1-9d55-c334780dd6fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:55.809063 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608e6a26-80c1-44e1-9d55-c334780dd6fb None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:55.809349 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608e6a26-80c1-44e1-9d55-c334780dd6fb None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:55.809630 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608e6a26-80c1-44e1-9d55-c334780dd6fb None admin] PATH_INFO: `/v3/projects/e04d4501005f4310819c6d8307f3df10` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:55.813810 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608e6a26-80c1-44e1-9d55-c334780dd6fb None admin] RBAC: Authorizing `identity:get_project(project_id=e04d4501005f4310819c6d8307f3df10)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:55.815285 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608e6a26-80c1-44e1-9d55-c334780dd6fb None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:55.819576 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 202/398] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jun 7 14:19:55 2022] GET /identity/v3/projects/e04d4501005f4310819c6d8307f3df10 => generated 298 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:55.822494 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608e6a26-80c1-44e1-9d55-c334780dd6fb None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:55.857105 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774f44cf-8fe3-4dc7-9495-0ca807acc995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:55.858010 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774f44cf-8fe3-4dc7-9495-0ca807acc995 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:55.858350 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774f44cf-8fe3-4dc7-9495-0ca807acc995 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:55.859999 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774f44cf-8fe3-4dc7-9495-0ca807acc995 None admin] PATH_INFO: `/v3/projects/e04d4501005f4310819c6d8307f3df10` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:55.862303 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774f44cf-8fe3-4dc7-9495-0ca807acc995 None admin] RBAC: Authorizing `identity:get_project(project_id=e04d4501005f4310819c6d8307f3df10)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:55.863791 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774f44cf-8fe3-4dc7-9495-0ca807acc995 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:55.867820 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 197/399] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jun 7 14:19:55 2022] GET /identity/v3/projects/e04d4501005f4310819c6d8307f3df10 => generated 298 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:55.872187 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774f44cf-8fe3-4dc7-9495-0ca807acc995 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:56.047980 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774f44cf-8fe3-4dc7-9495-0ca807acc995 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:56.061139 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:56.078533 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608e6a26-80c1-44e1-9d55-c334780dd6fb None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:56.088499 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672ad5cf-388b-46ab-84e7-c943d069475c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:56.090121 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672ad5cf-388b-46ab-84e7-c943d069475c None admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:56.090572 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672ad5cf-388b-46ab-84e7-c943d069475c None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:56.091065 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672ad5cf-388b-46ab-84e7-c943d069475c None admin] PATH_INFO: `/v3/projects/e04d4501005f4310819c6d8307f3df10/groups/ddcbeccb8288428d8c03e142e4970595/roles/3771341b3be641139e752aab690397a4` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:56.093970 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672ad5cf-388b-46ab-84e7-c943d069475c None admin] RBAC: Authorizing `identity:create_grant(project_id=e04d4501005f4310819c6d8307f3df10, group_id=ddcbeccb8288428d8c03e142e4970595, role_id=3771341b3be641139e752aab690397a4)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:56.096520 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672ad5cf-388b-46ab-84e7-c943d069475c None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:56.114974 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:56.121883 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e5c726-081d-4fe9-8e8e-5f32c447d509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:56.122777 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e5c726-081d-4fe9-8e8e-5f32c447d509 None admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:56.123121 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e5c726-081d-4fe9-8e8e-5f32c447d509 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:56.123404 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e5c726-081d-4fe9-8e8e-5f32c447d509 None admin] PATH_INFO: `/v3/projects/e04d4501005f4310819c6d8307f3df10/groups/ddcbeccb8288428d8c03e142e4970595/roles/494cd000d6884a179c334848f52a3a21` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:56.125681 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e5c726-081d-4fe9-8e8e-5f32c447d509 None admin] RBAC: Authorizing `identity:create_grant(project_id=e04d4501005f4310819c6d8307f3df10, group_id=ddcbeccb8288428d8c03e142e4970595, role_id=494cd000d6884a179c334848f52a3a21)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:56.127281 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e5c726-081d-4fe9-8e8e-5f32c447d509 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:56.192235 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 198/400] 10.222.0.85 () {58 vars in 1442 bytes} [Tue Jun 7 14:19:55 2022] PUT /identity/v3/projects/e04d4501005f4310819c6d8307f3df10/groups/ddcbeccb8288428d8c03e142e4970595/roles/3771341b3be641139e752aab690397a4 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:19:56.194010 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792a2c62-d909-43e1-8c79-cdcbbaa1e01f None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:56.194310 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792a2c62-d909-43e1-8c79-cdcbbaa1e01f None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:56.194592 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792a2c62-d909-43e1-8c79-cdcbbaa1e01f None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:56.196232 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 199/401] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:19:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:19:56.202126 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bf1a3a-01ba-4a97-ad7a-61553839c419 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:56.202399 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bf1a3a-01ba-4a97-ad7a-61553839c419 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:56.202682 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bf1a3a-01ba-4a97-ad7a-61553839c419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:56.223932 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-86bf1a3a-01ba-4a97-ad7a-61553839c419 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:19:56.225327 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-86bf1a3a-01ba-4a97-ad7a-61553839c419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:19:56.241335 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 203/402] 10.222.0.85 () {58 vars in 1442 bytes} [Tue Jun 7 14:19:55 2022] PUT /identity/v3/projects/e04d4501005f4310819c6d8307f3df10/groups/ddcbeccb8288428d8c03e142e4970595/roles/494cd000d6884a179c334848f52a3a21 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:19:56.246098 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb46576-9045-44d4-bb16-5a40429418cf None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:56.246664 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb46576-9045-44d4-bb16-5a40429418cf None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:56.247136 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb46576-9045-44d4-bb16-5a40429418cf None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:56.253078 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 204/403] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:19:56 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:19:56.260918 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c1d4a0-82af-431c-996c-1ec37a0fbf71 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:56.261824 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c1d4a0-82af-431c-996c-1ec37a0fbf71 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:56.262574 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c1d4a0-82af-431c-996c-1ec37a0fbf71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:56.298369 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-27c1d4a0-82af-431c-996c-1ec37a0fbf71 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:19:56.300011 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-27c1d4a0-82af-431c-996c-1ec37a0fbf71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:19:56.331635 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 200/404] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:19:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:19:56.359640 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bf1a3a-01ba-4a97-ad7a-61553839c419 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:56.444717 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 205/405] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:19:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 14:19:56.477918 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c1d4a0-82af-431c-996c-1ec37a0fbf71 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:56.594604 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:56.618836 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bf1a3a-01ba-4a97-ad7a-61553839c419 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:56.681449 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8462d1e-c842-48c3-89df-2f1729b90676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:56.683631 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8462d1e-c842-48c3-89df-2f1729b90676 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:56.683631 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8462d1e-c842-48c3-89df-2f1729b90676 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:56.683631 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8462d1e-c842-48c3-89df-2f1729b90676 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:56.703382 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8462d1e-c842-48c3-89df-2f1729b90676 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:19:56.720292 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-e8462d1e-c842-48c3-89df-2f1729b90676 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 14:19:56.720292 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:19:56.720292 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:19:56.720292 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:19:56.720292 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:19:56.720292 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:19:56.720292 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:19:56.720292 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:19:56.720292 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:19:56.720292 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:19:56.720292 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:19:56.720292 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:19:56.720292 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:19:56.720292 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:19:56.720292 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:19:56.720292 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:19:56.721290 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:19:56.721290 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:19:56.721290 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:19:56.721290 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:19:56.721290 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:19:56.721290 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:19:56.721290 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:19:56.721290 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:19:56.721290 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:19:56.721290 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:19:56.721290 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:19:56.721290 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:19:56.721290 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:19:56.721290 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:19:56.721290 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:19:56.721290 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:19:56.722130 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:19:56.722130 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:19:56.722130 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:19:56.722130 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:19:56.722130 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:19:56.722130 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:19:56.722130 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:19:56.722130 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:19:56.722130 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:19:56.722130 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:19:56.722130 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 14:19:56.722130 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:19:56.722861 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 201/406] 10.222.0.85 () {56 vars in 1114 bytes} [Tue Jun 7 14:19:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:19:56.731372 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8462d1e-c842-48c3-89df-2f1729b90676 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:56.746603 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c1d4a0-82af-431c-996c-1ec37a0fbf71 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:56.792749 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:56.833770 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8d50b6-eafa-41b5-be45-c4b3526b1558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:56.834592 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d50b6-eafa-41b5-be45-c4b3526b1558 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:56.834875 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d50b6-eafa-41b5-be45-c4b3526b1558 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:56.835151 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d50b6-eafa-41b5-be45-c4b3526b1558 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:56.855311 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8d50b6-eafa-41b5-be45-c4b3526b1558 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:19:56.863325 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-8b8d50b6-eafa-41b5-be45-c4b3526b1558 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:19:56.863325 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:19:56.863325 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:19:56.863325 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:19:56.863325 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:19:56.863325 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:19:56.863325 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:19:56.863325 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:19:56.863325 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:19:56.863325 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:19:56.863325 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:19:56.863325 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:19:56.863325 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:19:56.863325 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:19:56.863325 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:19:56.863325 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:19:56.864262 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:19:56.864262 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:19:56.864262 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:19:56.864262 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:19:56.864262 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:19:56.864262 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:19:56.864262 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:19:56.864262 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:19:56.864262 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:19:56.864262 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:19:56.864262 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:19:56.864262 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:19:56.864262 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:19:56.864262 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:19:56.864262 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:19:56.864262 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:19:56.866136 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:19:56.866136 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:19:56.866136 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:19:56.866136 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:19:56.866136 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:19:56.866136 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:19:56.866136 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:19:56.866136 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:19:56.866136 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:19:56.866136 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:19:56.866136 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:19:56.866136 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:19:56.866136 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 206/407] 10.222.0.85 () {56 vars in 1096 bytes} [Tue Jun 7 14:19:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 389 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:19:56.874536 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8d50b6-eafa-41b5-be45-c4b3526b1558 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:56.922103 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8462d1e-c842-48c3-89df-2f1729b90676 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:56.971669 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55902b23-5548-4f04-aa01-aaae3c330170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:56.972292 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55902b23-5548-4f04-aa01-aaae3c330170 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:56.972615 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55902b23-5548-4f04-aa01-aaae3c330170 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:56.972898 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55902b23-5548-4f04-aa01-aaae3c330170 None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:56.973718 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55902b23-5548-4f04-aa01-aaae3c330170 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:19:56.974030 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55902b23-5548-4f04-aa01-aaae3c330170 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:56.975459 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55902b23-5548-4f04-aa01-aaae3c330170 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:56.986066 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 202/408] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jun 7 14:19:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:56.991322 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55902b23-5548-4f04-aa01-aaae3c330170 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:57.013363 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:57.112879 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55902b23-5548-4f04-aa01-aaae3c330170 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:57.156160 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cdb059-beaa-4503-82ff-d19171f25413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:57.157344 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cdb059-beaa-4503-82ff-d19171f25413 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:57.157709 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cdb059-beaa-4503-82ff-d19171f25413 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:57.158116 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cdb059-beaa-4503-82ff-d19171f25413 None admin] PATH_INFO: `/v3/groups/ddcbeccb8288428d8c03e142e4970595` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:57.159647 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cdb059-beaa-4503-82ff-d19171f25413 None admin] RBAC: Authorizing `identity:get_group(group_id=ddcbeccb8288428d8c03e142e4970595)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:57.161279 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cdb059-beaa-4503-82ff-d19171f25413 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:57.162372 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8d50b6-eafa-41b5-be45-c4b3526b1558 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:57.163599 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 203/409] 10.222.0.85 () {56 vars in 1180 bytes} [Tue Jun 7 14:19:56 2022] GET /identity/v3/groups/ddcbeccb8288428d8c03e142e4970595 => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:57.169442 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cdb059-beaa-4503-82ff-d19171f25413 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:57.201659 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7d3e46-a1fa-4f36-a977-3b89e496b5b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:57.202345 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7d3e46-a1fa-4f36-a977-3b89e496b5b2 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:57.202632 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7d3e46-a1fa-4f36-a977-3b89e496b5b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:57.202912 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7d3e46-a1fa-4f36-a977-3b89e496b5b2 None admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:57.203681 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7d3e46-a1fa-4f36-a977-3b89e496b5b2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:19:57.203994 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7d3e46-a1fa-4f36-a977-3b89e496b5b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:57.205475 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7d3e46-a1fa-4f36-a977-3b89e496b5b2 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:57.215917 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 207/410] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jun 7 14:19:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:19:57.221662 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7d3e46-a1fa-4f36-a977-3b89e496b5b2 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:57.285787 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:57.356094 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7d3e46-a1fa-4f36-a977-3b89e496b5b2 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:57.364876 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:57.397773 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a7d7b4-6bc1-4bfc-b3c5-50ed53b18e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:57.401309 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a7d7b4-6bc1-4bfc-b3c5-50ed53b18e2c None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:57.403419 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cdb059-beaa-4503-82ff-d19171f25413 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:57.403898 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a7d7b4-6bc1-4bfc-b3c5-50ed53b18e2c None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:57.404342 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a7d7b4-6bc1-4bfc-b3c5-50ed53b18e2c None admin] PATH_INFO: `/v3/groups/c1f2253e8d3a4fff9f5f3e3bbc9a247f` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:57.409503 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a7d7b4-6bc1-4bfc-b3c5-50ed53b18e2c None admin] RBAC: Authorizing `identity:get_group(group_id=c1f2253e8d3a4fff9f5f3e3bbc9a247f)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:57.412174 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a7d7b4-6bc1-4bfc-b3c5-50ed53b18e2c None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:57.416245 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 208/411] 10.222.0.85 () {56 vars in 1180 bytes} [Tue Jun 7 14:19:57 2022] GET /identity/v3/groups/c1f2253e8d3a4fff9f5f3e3bbc9a247f => generated 229 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:19:57.426049 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a7d7b4-6bc1-4bfc-b3c5-50ed53b18e2c None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:57.471399 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5e74f7-64d1-414d-975c-456a4a3f4348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:57.473256 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5e74f7-64d1-414d-975c-456a4a3f4348 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:57.475078 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5e74f7-64d1-414d-975c-456a4a3f4348 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:57.475564 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5e74f7-64d1-414d-975c-456a4a3f4348 None admin] PATH_INFO: `/v3/projects/f95de357ccad493984042656b2c0689a` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:57.479205 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5e74f7-64d1-414d-975c-456a4a3f4348 None admin] RBAC: Authorizing `identity:get_project(project_id=f95de357ccad493984042656b2c0689a)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:57.481770 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5e74f7-64d1-414d-975c-456a4a3f4348 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:57.484543 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 204/412] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jun 7 14:19:57 2022] GET /identity/v3/projects/f95de357ccad493984042656b2c0689a => generated 302 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:57.490289 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5e74f7-64d1-414d-975c-456a4a3f4348 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:57.645604 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5e74f7-64d1-414d-975c-456a4a3f4348 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:57.648940 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a7d7b4-6bc1-4bfc-b3c5-50ed53b18e2c None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:57.667801 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:57.696730 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3011aa2d-fa91-492a-a283-f60642634dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:57.698045 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3011aa2d-fa91-492a-a283-f60642634dfc None admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:57.698533 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3011aa2d-fa91-492a-a283-f60642634dfc None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:57.700419 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3011aa2d-fa91-492a-a283-f60642634dfc None admin] PATH_INFO: `/v3/projects/f95de357ccad493984042656b2c0689a/groups/ddcbeccb8288428d8c03e142e4970595/roles/494cd000d6884a179c334848f52a3a21` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:57.711601 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3011aa2d-fa91-492a-a283-f60642634dfc None admin] RBAC: Authorizing `identity:create_grant(project_id=f95de357ccad493984042656b2c0689a, group_id=ddcbeccb8288428d8c03e142e4970595, role_id=494cd000d6884a179c334848f52a3a21)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:57.714569 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3011aa2d-fa91-492a-a283-f60642634dfc None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:57.726810 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d811a4-c9e9-4458-b179-67a3029f115d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:57.728683 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d811a4-c9e9-4458-b179-67a3029f115d None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:57.731294 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d811a4-c9e9-4458-b179-67a3029f115d None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:57.732151 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d811a4-c9e9-4458-b179-67a3029f115d None admin] PATH_INFO: `/v3/projects/61703df7c02543d6ba1f949d9811ee43` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:57.734963 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d811a4-c9e9-4458-b179-67a3029f115d None admin] RBAC: Authorizing `identity:get_project(project_id=61703df7c02543d6ba1f949d9811ee43)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:57.738258 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d811a4-c9e9-4458-b179-67a3029f115d None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:57.742868 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 209/413] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jun 7 14:19:57 2022] GET /identity/v3/projects/61703df7c02543d6ba1f949d9811ee43 => generated 344 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:57.749073 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d811a4-c9e9-4458-b179-67a3029f115d None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:57.821018 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 205/414] 10.222.0.85 () {58 vars in 1442 bytes} [Tue Jun 7 14:19:57 2022] PUT /identity/v3/projects/f95de357ccad493984042656b2c0689a/groups/ddcbeccb8288428d8c03e142e4970595/roles/494cd000d6884a179c334848f52a3a21 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:19:57.848526 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d1abbd-4fb5-4a52-83ad-70ed9d09f5a1 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:57.849381 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d1abbd-4fb5-4a52-83ad-70ed9d09f5a1 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:57.850129 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d1abbd-4fb5-4a52-83ad-70ed9d09f5a1 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:57.853165 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 206/415] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:19:57 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 14:19:57.860279 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4461eb95-c9b4-4f95-b358-e19b2dea42ad None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:57.861333 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4461eb95-c9b4-4f95-b358-e19b2dea42ad None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:57.862067 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4461eb95-c9b4-4f95-b358-e19b2dea42ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:57.923014 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-4461eb95-c9b4-4f95-b358-e19b2dea42ad None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:19:57.925526 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-4461eb95-c9b4-4f95-b358-e19b2dea42ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:19:57.944060 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:58.035589 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d811a4-c9e9-4458-b179-67a3029f115d None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:58.077763 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 207/416] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:19:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 14:19:58.100946 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe279466-3840-447a-b88a-905ee34cc321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:58.103074 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe279466-3840-447a-b88a-905ee34cc321 None admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:58.103515 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe279466-3840-447a-b88a-905ee34cc321 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:58.104007 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe279466-3840-447a-b88a-905ee34cc321 None admin] PATH_INFO: `/v3/projects/61703df7c02543d6ba1f949d9811ee43/groups/c1f2253e8d3a4fff9f5f3e3bbc9a247f/roles/9baff2b18f2649f7b757ff2e1f63c853` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:58.104306 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4461eb95-c9b4-4f95-b358-e19b2dea42ad None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:58.107558 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe279466-3840-447a-b88a-905ee34cc321 None admin] RBAC: Authorizing `identity:create_grant(project_id=61703df7c02543d6ba1f949d9811ee43, group_id=c1f2253e8d3a4fff9f5f3e3bbc9a247f, role_id=9baff2b18f2649f7b757ff2e1f63c853)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:58.110914 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe279466-3840-447a-b88a-905ee34cc321 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:58.187700 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 210/417] 10.222.0.85 () {58 vars in 1442 bytes} [Tue Jun 7 14:19:57 2022] PUT /identity/v3/projects/61703df7c02543d6ba1f949d9811ee43/groups/c1f2253e8d3a4fff9f5f3e3bbc9a247f/roles/9baff2b18f2649f7b757ff2e1f63c853 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:19:58.235367 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4461eb95-c9b4-4f95-b358-e19b2dea42ad None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:58.280402 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9462817-c5ce-4d5d-b521-65b668dd4876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:58.281227 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9462817-c5ce-4d5d-b521-65b668dd4876 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:58.281510 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9462817-c5ce-4d5d-b521-65b668dd4876 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:58.281787 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9462817-c5ce-4d5d-b521-65b668dd4876 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:58.297872 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9462817-c5ce-4d5d-b521-65b668dd4876 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:19:58.305903 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-b9462817-c5ce-4d5d-b521-65b668dd4876 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 14:19:58.305903 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:19:58.305903 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:19:58.305903 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:19:58.305903 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:19:58.305903 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:19:58.305903 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:19:58.305903 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:19:58.305903 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:19:58.305903 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:19:58.305903 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:19:58.305903 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:19:58.305903 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:19:58.305903 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:19:58.305903 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:19:58.305903 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:19:58.306828 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:19:58.306828 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:19:58.306828 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:19:58.306828 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:19:58.306828 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:19:58.306828 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:19:58.306828 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:19:58.306828 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:19:58.306828 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:19:58.306828 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:19:58.306828 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:19:58.306828 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:19:58.306828 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:19:58.306828 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:19:58.306828 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:19:58.306828 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:19:58.307608 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:19:58.307608 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:19:58.307608 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:19:58.307608 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:19:58.307608 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:19:58.307608 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:19:58.307608 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:19:58.307608 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:19:58.307608 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:19:58.307608 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:19:58.307608 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 14:19:58.307608 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:19:58.307608 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 208/418] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jun 7 14:19:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:19:58.313545 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe279466-3840-447a-b88a-905ee34cc321 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:58.383530 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:58.472531 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe279466-3840-447a-b88a-905ee34cc321 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:58.512319 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a578bd-b79d-458f-8c7c-a4de3e66c00c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:58.513041 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a578bd-b79d-458f-8c7c-a4de3e66c00c None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:58.513324 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a578bd-b79d-458f-8c7c-a4de3e66c00c None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:58.513604 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a578bd-b79d-458f-8c7c-a4de3e66c00c None admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:58.514377 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a578bd-b79d-458f-8c7c-a4de3e66c00c None admin] RBAC: Adding query filter params (name=member) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:19:58.514682 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a578bd-b79d-458f-8c7c-a4de3e66c00c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:58.516203 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a578bd-b79d-458f-8c7c-a4de3e66c00c None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:58.527007 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 211/419] 10.222.0.85 () {56 vars in 1101 bytes} [Tue Jun 7 14:19:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:58.533537 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9462817-c5ce-4d5d-b521-65b668dd4876 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:58.539294 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:58.704556 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9462817-c5ce-4d5d-b521-65b668dd4876 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:58.748475 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6733821-299e-418f-950c-a418bd22fcf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:58.749486 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6733821-299e-418f-950c-a418bd22fcf5 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:58.749999 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6733821-299e-418f-950c-a418bd22fcf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:58.750478 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6733821-299e-418f-950c-a418bd22fcf5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:58.834872 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6733821-299e-418f-950c-a418bd22fcf5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:58.837293 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6733821-299e-418f-950c-a418bd22fcf5 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:58.840682 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:58.991754 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 209/420] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jun 7 14:19:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 14:19:58.992896 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a578bd-b79d-458f-8c7c-a4de3e66c00c None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:59.114329 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:59.118374 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a578bd-b79d-458f-8c7c-a4de3e66c00c None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:59.158701 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4403f3f1-0cff-4052-a58c-b2efe4f1f386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:59.159960 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4403f3f1-0cff-4052-a58c-b2efe4f1f386 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:59.160496 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4403f3f1-0cff-4052-a58c-b2efe4f1f386 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:59.160971 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4403f3f1-0cff-4052-a58c-b2efe4f1f386 None admin] PATH_INFO: `/v3/projects/f95de357ccad493984042656b2c0689a` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:59.162408 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4403f3f1-0cff-4052-a58c-b2efe4f1f386 None admin] RBAC: Authorizing `identity:get_project(project_id=f95de357ccad493984042656b2c0689a)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:59.164142 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4403f3f1-0cff-4052-a58c-b2efe4f1f386 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:59.166441 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 212/421] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jun 7 14:19:58 2022] GET /identity/v3/projects/f95de357ccad493984042656b2c0689a => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:59.172139 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6733821-299e-418f-950c-a418bd22fcf5 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:59.187264 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd565d8-27db-4744-a3d1-b583c2fef2c9 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:59.187587 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd565d8-27db-4744-a3d1-b583c2fef2c9 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:59.187845 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd565d8-27db-4744-a3d1-b583c2fef2c9 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:59.191102 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 213/422] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:19:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 14:19:59.196131 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d9a087-926d-4cef-bc1a-7d9c100f0b3f None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:59.196481 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d9a087-926d-4cef-bc1a-7d9c100f0b3f None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:59.196722 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d9a087-926d-4cef-bc1a-7d9c100f0b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:59.217658 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-f0d9a087-926d-4cef-bc1a-7d9c100f0b3f None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:19:59.218879 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-f0d9a087-926d-4cef-bc1a-7d9c100f0b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:19:59.316528 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 214/423] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:19:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 14:19:59.336781 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f9e6df-e181-492e-98ac-58a7d5dc79f4 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:59.337081 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f9e6df-e181-492e-98ac-58a7d5dc79f4 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:59.337352 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f9e6df-e181-492e-98ac-58a7d5dc79f4 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:59.338776 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 215/424] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:19:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 14:19:59.339458 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f9e6df-e181-492e-98ac-58a7d5dc79f4 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:59.367421 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6733821-299e-418f-950c-a418bd22fcf5 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:59.369946 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:59.425472 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6083c7-6ad3-4c3f-b9c6-544fe2e9447c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:59.427472 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6083c7-6ad3-4c3f-b9c6-544fe2e9447c None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:59.428519 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6083c7-6ad3-4c3f-b9c6-544fe2e9447c None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:59.429981 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6083c7-6ad3-4c3f-b9c6-544fe2e9447c None admin] PATH_INFO: `/v3/groups/ddcbeccb8288428d8c03e142e4970595` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:59.432027 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6083c7-6ad3-4c3f-b9c6-544fe2e9447c None admin] RBAC: Authorizing `identity:get_group(group_id=ddcbeccb8288428d8c03e142e4970595)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:59.434712 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6083c7-6ad3-4c3f-b9c6-544fe2e9447c None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:59.436695 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 210/425] 10.222.0.85 () {56 vars in 1180 bytes} [Tue Jun 7 14:19:59 2022] GET /identity/v3/groups/ddcbeccb8288428d8c03e142e4970595 => generated 226 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:59.442395 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac184a67-0cdf-4917-af2e-53b04e3ce6e3 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:59.442395 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac184a67-0cdf-4917-af2e-53b04e3ce6e3 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:59.442395 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac184a67-0cdf-4917-af2e-53b04e3ce6e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:59.471244 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-ac184a67-0cdf-4917-af2e-53b04e3ce6e3 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:19:59.474042 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-ac184a67-0cdf-4917-af2e-53b04e3ce6e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:19:59.491391 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:59.498377 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f9e6df-e181-492e-98ac-58a7d5dc79f4 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:59.541528 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1ee2fa-235e-4737-a517-babac041a7b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:59.542856 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ee2fa-235e-4737-a517-babac041a7b7 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:59.543361 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ee2fa-235e-4737-a517-babac041a7b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:59.543859 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ee2fa-235e-4737-a517-babac041a7b7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:59.558691 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1ee2fa-235e-4737-a517-babac041a7b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:19:59.564518 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 211/426] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:19:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:19:59.565482 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac184a67-0cdf-4917-af2e-53b04e3ce6e3 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:59.567709 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-7d1ee2fa-235e-4737-a517-babac041a7b7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 14:19:59.567709 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:19:59.567709 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:19:59.567709 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:19:59.567709 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:19:59.567709 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:19:59.567709 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:19:59.567709 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:19:59.567709 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:19:59.567709 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:19:59.567709 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:19:59.567709 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:19:59.567709 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:19:59.567709 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:19:59.567709 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:19:59.567709 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:19:59.568622 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:19:59.568622 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:19:59.568622 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:19:59.568622 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:19:59.568622 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:19:59.568622 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:19:59.568622 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:19:59.568622 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:19:59.568622 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:19:59.568622 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:19:59.568622 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:19:59.568622 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:19:59.568622 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:19:59.568622 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:19:59.568622 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:19:59.568622 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:19:59.569397 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:19:59.569397 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:19:59.569397 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:19:59.569397 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:19:59.569397 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:19:59.569397 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:19:59.569397 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:19:59.569397 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:19:59.569397 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:19:59.569397 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:19:59.569397 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 14:19:59.569397 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:19:59.572720 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 216/427] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jun 7 14:19:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:19:59.583971 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1ee2fa-235e-4737-a517-babac041a7b7 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:59.702737 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:19:59.709880 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac184a67-0cdf-4917-af2e-53b04e3ce6e3 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:59.770504 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d016f69d-12ea-4d9e-b832-67368f6afc2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:59.772657 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d016f69d-12ea-4d9e-b832-67368f6afc2f None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:59.773742 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d016f69d-12ea-4d9e-b832-67368f6afc2f None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:59.774220 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d016f69d-12ea-4d9e-b832-67368f6afc2f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:59.775784 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d016f69d-12ea-4d9e-b832-67368f6afc2f None admin] RBAC: Adding query filter params (group.id=ddcbeccb8288428d8c03e142e4970595, role.id=3771341b3be641139e752aab690397a4, scope.project.id=f95de357ccad493984042656b2c0689a) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:19:59.776386 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d016f69d-12ea-4d9e-b832-67368f6afc2f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:59.779096 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d016f69d-12ea-4d9e-b832-67368f6afc2f None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:59.790635 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 212/428] 10.222.0.85 () {56 vars in 1376 bytes} [Tue Jun 7 14:19:59 2022] GET /identity/v3/role_assignments?group.id=ddcbeccb8288428d8c03e142e4970595&scope.project.id=f95de357ccad493984042656b2c0689a&role.id=3771341b3be641139e752aab690397a4 => generated 611 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:59.792139 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d016f69d-12ea-4d9e-b832-67368f6afc2f None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:19:59.835474 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1ee2fa-235e-4737-a517-babac041a7b7 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:19:59.907453 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42ee840-0677-44b8-aba1-95347ddca24e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:19:59.908234 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42ee840-0677-44b8-aba1-95347ddca24e None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:19:59.908573 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42ee840-0677-44b8-aba1-95347ddca24e None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:19:59.908858 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42ee840-0677-44b8-aba1-95347ddca24e None admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:19:59.909675 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42ee840-0677-44b8-aba1-95347ddca24e None admin] RBAC: Adding query filter params (name=member) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:19:59.909988 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42ee840-0677-44b8-aba1-95347ddca24e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:19:59.911439 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42ee840-0677-44b8-aba1-95347ddca24e None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:19:59.940535 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 217/429] 10.222.0.85 () {56 vars in 1101 bytes} [Tue Jun 7 14:19:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:19:59.968101 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42ee840-0677-44b8-aba1-95347ddca24e None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:00.029649 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:00.043685 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d016f69d-12ea-4d9e-b832-67368f6afc2f None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:00.124755 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d137b88c-53a7-4135-ac57-f3ba3234ab50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:00.126084 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d137b88c-53a7-4135-ac57-f3ba3234ab50 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:00.126569 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d137b88c-53a7-4135-ac57-f3ba3234ab50 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:00.127048 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d137b88c-53a7-4135-ac57-f3ba3234ab50 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:00.170730 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d137b88c-53a7-4135-ac57-f3ba3234ab50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:20:00.180351 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-d137b88c-53a7-4135-ac57-f3ba3234ab50 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 14:20:00.180351 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:00.180351 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:00.180351 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:00.180351 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:00.180351 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:00.180351 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:00.180351 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:00.180351 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:00.180351 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:00.180351 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:00.180351 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:00.180351 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:20:00.180351 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:20:00.180351 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:20:00.180351 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:20:00.181304 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:20:00.181304 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:20:00.181304 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:20:00.181304 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:20:00.181304 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:00.181304 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:00.181304 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:00.181304 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:00.181304 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:00.181304 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:00.181304 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:00.181304 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:00.181304 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:00.181304 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:00.181304 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:00.181304 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:00.182103 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:00.182103 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:00.182103 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:00.182103 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:00.182103 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:20:00.182103 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:20:00.182103 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:20:00.182103 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:20:00.182103 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:20:00.182103 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:20:00.182103 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 14:20:00.182103 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:00.186041 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 213/430] 10.222.0.85 () {56 vars in 1114 bytes} [Tue Jun 7 14:19:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 395 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:00.192735 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d137b88c-53a7-4135-ac57-f3ba3234ab50 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:00.251615 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42ee840-0677-44b8-aba1-95347ddca24e None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:00.259812 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:00.300058 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5225383-f9c1-4bdc-b05d-502a5ed62639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:00.301069 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5225383-f9c1-4bdc-b05d-502a5ed62639 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:00.301430 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5225383-f9c1-4bdc-b05d-502a5ed62639 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:00.301781 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5225383-f9c1-4bdc-b05d-502a5ed62639 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:00.348802 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d137b88c-53a7-4135-ac57-f3ba3234ab50 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:00.397673 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae043ab-3ff3-4d0f-b6e4-16d7ae9389b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:00.401420 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae043ab-3ff3-4d0f-b6e4-16d7ae9389b1 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:00.401538 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5225383-f9c1-4bdc-b05d-502a5ed62639 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:00.401538 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5225383-f9c1-4bdc-b05d-502a5ed62639 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:00.402493 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae043ab-3ff3-4d0f-b6e4-16d7ae9389b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:00.403019 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae043ab-3ff3-4d0f-b6e4-16d7ae9389b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:00.404011 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae043ab-3ff3-4d0f-b6e4-16d7ae9389b1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:00.404622 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae043ab-3ff3-4d0f-b6e4-16d7ae9389b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:00.406241 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae043ab-3ff3-4d0f-b6e4-16d7ae9389b1 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:00.420013 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 214/431] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jun 7 14:20:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:20:00.427103 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae043ab-3ff3-4d0f-b6e4-16d7ae9389b1 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:00.428915 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:00.545116 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:00.548825 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 218/432] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jun 7 14:19:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 581 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 14:20:00.555178 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5225383-f9c1-4bdc-b05d-502a5ed62639 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:00.572975 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae043ab-3ff3-4d0f-b6e4-16d7ae9389b1 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:00.627280 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a990bc-864e-4a6c-a8b5-76a888b30334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:00.628826 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a990bc-864e-4a6c-a8b5-76a888b30334 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:00.629394 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a990bc-864e-4a6c-a8b5-76a888b30334 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:00.630084 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a990bc-864e-4a6c-a8b5-76a888b30334 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:00.678045 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5225383-f9c1-4bdc-b05d-502a5ed62639 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:00.702994 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:00.720530 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a990bc-864e-4a6c-a8b5-76a888b30334 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:00.721497 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2b5aa2-fe2e-4b6e-96c9-ee2d47fd5ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:00.724887 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2b5aa2-fe2e-4b6e-96c9-ee2d47fd5ca2 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:00.725042 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a990bc-864e-4a6c-a8b5-76a888b30334 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:00.725124 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2b5aa2-fe2e-4b6e-96c9-ee2d47fd5ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:00.725124 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2b5aa2-fe2e-4b6e-96c9-ee2d47fd5ca2 None admin] PATH_INFO: `/v3/projects/e04d4501005f4310819c6d8307f3df10` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:00.726545 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2b5aa2-fe2e-4b6e-96c9-ee2d47fd5ca2 None admin] RBAC: Authorizing `identity:get_project(project_id=e04d4501005f4310819c6d8307f3df10)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:00.729269 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2b5aa2-fe2e-4b6e-96c9-ee2d47fd5ca2 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:00.736958 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 219/433] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jun 7 14:20:00 2022] GET /identity/v3/projects/e04d4501005f4310819c6d8307f3df10 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:20:00.738078 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2b5aa2-fe2e-4b6e-96c9-ee2d47fd5ca2 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:00.827566 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:00.852684 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 215/434] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jun 7 14:20:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 14:20:00.854416 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a990bc-864e-4a6c-a8b5-76a888b30334 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:00.855441 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2b5aa2-fe2e-4b6e-96c9-ee2d47fd5ca2 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:00.893283 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646e4104-ad58-4706-a943-ebef804c5b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:00.894164 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646e4104-ad58-4706-a943-ebef804c5b33 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:00.894451 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646e4104-ad58-4706-a943-ebef804c5b33 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:00.894735 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646e4104-ad58-4706-a943-ebef804c5b33 None admin] PATH_INFO: `/v3/groups/ddcbeccb8288428d8c03e142e4970595` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:00.895969 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646e4104-ad58-4706-a943-ebef804c5b33 None admin] RBAC: Authorizing `identity:get_group(group_id=ddcbeccb8288428d8c03e142e4970595)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:00.897468 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646e4104-ad58-4706-a943-ebef804c5b33 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:00.904186 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 220/435] 10.222.0.85 () {56 vars in 1180 bytes} [Tue Jun 7 14:20:00 2022] GET /identity/v3/groups/ddcbeccb8288428d8c03e142e4970595 => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:00.905254 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646e4104-ad58-4706-a943-ebef804c5b33 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:00.978688 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:00.985237 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a990bc-864e-4a6c-a8b5-76a888b30334 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:01.025698 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7397a81b-fc7b-47bd-819d-4c8839d67022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:01.026953 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7397a81b-fc7b-47bd-819d-4c8839d67022 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:01.027443 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7397a81b-fc7b-47bd-819d-4c8839d67022 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:01.027920 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7397a81b-fc7b-47bd-819d-4c8839d67022 None admin] PATH_INFO: `/v3/projects/e04d4501005f4310819c6d8307f3df10` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:01.028931 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646e4104-ad58-4706-a943-ebef804c5b33 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:01.029662 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7397a81b-fc7b-47bd-819d-4c8839d67022 None admin] RBAC: Authorizing `identity:get_project(project_id=e04d4501005f4310819c6d8307f3df10)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:01.031586 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7397a81b-fc7b-47bd-819d-4c8839d67022 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:01.038085 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 216/436] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jun 7 14:20:00 2022] GET /identity/v3/projects/e04d4501005f4310819c6d8307f3df10 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:20:01.039386 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7397a81b-fc7b-47bd-819d-4c8839d67022 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:01.068544 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52ef2d0-ea4c-4745-9f6c-b7444f64ebd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:01.068999 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52ef2d0-ea4c-4745-9f6c-b7444f64ebd0 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:01.069290 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52ef2d0-ea4c-4745-9f6c-b7444f64ebd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:01.069576 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52ef2d0-ea4c-4745-9f6c-b7444f64ebd0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:01.071087 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52ef2d0-ea4c-4745-9f6c-b7444f64ebd0 None admin] RBAC: Adding query filter params (group.id=ddcbeccb8288428d8c03e142e4970595, role.id=3771341b3be641139e752aab690397a4, scope.project.id=e04d4501005f4310819c6d8307f3df10) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:01.071400 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52ef2d0-ea4c-4745-9f6c-b7444f64ebd0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:01.072981 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52ef2d0-ea4c-4745-9f6c-b7444f64ebd0 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:01.080110 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 221/437] 10.222.0.85 () {56 vars in 1376 bytes} [Tue Jun 7 14:20:00 2022] GET /identity/v3/role_assignments?group.id=ddcbeccb8288428d8c03e142e4970595&scope.project.id=e04d4501005f4310819c6d8307f3df10&role.id=3771341b3be641139e752aab690397a4 => generated 611 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:20:01.081295 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d46d2b8-59e4-4834-9514-57efbf6888dc None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:01.081587 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d46d2b8-59e4-4834-9514-57efbf6888dc None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:01.081864 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d46d2b8-59e4-4834-9514-57efbf6888dc None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:01.083516 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 222/438] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:20:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:20:01.092776 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dab93c-cdbd-4fb8-99ae-db4d7263d277 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:01.092776 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dab93c-cdbd-4fb8-99ae-db4d7263d277 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:01.093073 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dab93c-cdbd-4fb8-99ae-db4d7263d277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:01.122497 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-00dab93c-cdbd-4fb8-99ae-db4d7263d277 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:20:01.123767 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-00dab93c-cdbd-4fb8-99ae-db4d7263d277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:20:01.124896 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:01.203603 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7397a81b-fc7b-47bd-819d-4c8839d67022 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:01.223398 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 223/439] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:20:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:20:01.247148 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a232ce30-43d5-4450-8e64-6361dcc0d400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:01.247951 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dab93c-cdbd-4fb8-99ae-db4d7263d277 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:01.248435 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a232ce30-43d5-4450-8e64-6361dcc0d400 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:01.248724 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a232ce30-43d5-4450-8e64-6361dcc0d400 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:01.249004 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a232ce30-43d5-4450-8e64-6361dcc0d400 None admin] PATH_INFO: `/v3/groups/ddcbeccb8288428d8c03e142e4970595` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:01.250313 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a232ce30-43d5-4450-8e64-6361dcc0d400 None admin] RBAC: Authorizing `identity:get_group(group_id=ddcbeccb8288428d8c03e142e4970595)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:01.252793 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a232ce30-43d5-4450-8e64-6361dcc0d400 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:01.254886 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 217/440] 10.222.0.85 () {56 vars in 1180 bytes} [Tue Jun 7 14:20:01 2022] GET /identity/v3/groups/ddcbeccb8288428d8c03e142e4970595 => generated 226 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:01.260829 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a232ce30-43d5-4450-8e64-6361dcc0d400 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:01.318975 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:01.388587 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dab93c-cdbd-4fb8-99ae-db4d7263d277 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:01.436710 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7611cfd5-4f18-420b-a2da-ad3a336ff874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:01.437983 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7611cfd5-4f18-420b-a2da-ad3a336ff874 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:01.438131 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a232ce30-43d5-4450-8e64-6361dcc0d400 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:01.438317 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7611cfd5-4f18-420b-a2da-ad3a336ff874 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:01.438571 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7611cfd5-4f18-420b-a2da-ad3a336ff874 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:01.455318 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7611cfd5-4f18-420b-a2da-ad3a336ff874 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:20:01.464000 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-7611cfd5-4f18-420b-a2da-ad3a336ff874 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 14:20:01.464000 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:01.464000 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:01.464000 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:01.464000 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:01.464000 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:01.464000 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:01.464000 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:01.464000 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:01.464000 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:01.464000 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:01.464000 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:01.464000 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:20:01.464000 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:20:01.464000 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:20:01.464000 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:20:01.464935 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:20:01.464935 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:20:01.464935 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:20:01.464935 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:20:01.464935 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:01.464935 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:01.464935 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:01.464935 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:01.464935 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:01.464935 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:01.464935 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:01.464935 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:01.464935 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:01.464935 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:01.464935 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:01.464935 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:01.465740 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:01.465740 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:01.465740 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:01.465740 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:01.465740 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:20:01.465740 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:20:01.465740 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:20:01.465740 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:20:01.465740 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:20:01.465740 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:20:01.465740 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 14:20:01.465740 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:01.465740 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 224/441] 10.222.0.85 () {56 vars in 1114 bytes} [Tue Jun 7 14:20:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:20:01.471329 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7611cfd5-4f18-420b-a2da-ad3a336ff874 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:01.495595 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:01.510413 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d959e4d0-7e7e-459e-b990-56e5dba4cbcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:01.511112 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d959e4d0-7e7e-459e-b990-56e5dba4cbcb None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:01.511376 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d959e4d0-7e7e-459e-b990-56e5dba4cbcb None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:01.511655 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d959e4d0-7e7e-459e-b990-56e5dba4cbcb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:01.512529 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d959e4d0-7e7e-459e-b990-56e5dba4cbcb None admin] RBAC: Adding query filter params (group.id=ddcbeccb8288428d8c03e142e4970595, role.id=494cd000d6884a179c334848f52a3a21, scope.project.id=e04d4501005f4310819c6d8307f3df10) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:01.512843 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d959e4d0-7e7e-459e-b990-56e5dba4cbcb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:01.516184 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d959e4d0-7e7e-459e-b990-56e5dba4cbcb None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:01.523854 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 218/442] 10.222.0.85 () {56 vars in 1376 bytes} [Tue Jun 7 14:20:01 2022] GET /identity/v3/role_assignments?group.id=ddcbeccb8288428d8c03e142e4970595&scope.project.id=e04d4501005f4310819c6d8307f3df10&role.id=494cd000d6884a179c334848f52a3a21 => generated 611 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:20:01.526605 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3e11c3-8bf8-4968-aa49-59ec8ebb2351 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:01.527083 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3e11c3-8bf8-4968-aa49-59ec8ebb2351 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:01.527533 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3e11c3-8bf8-4968-aa49-59ec8ebb2351 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:01.529935 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 219/443] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:20:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:20:01.540848 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c83d67d-d722-42d1-8b2b-6fa4892cd1ce None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:01.542260 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c83d67d-d722-42d1-8b2b-6fa4892cd1ce None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:01.542260 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c83d67d-d722-42d1-8b2b-6fa4892cd1ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:01.566702 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-0c83d67d-d722-42d1-8b2b-6fa4892cd1ce None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:20:01.567966 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-0c83d67d-d722-42d1-8b2b-6fa4892cd1ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:20:01.624163 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7611cfd5-4f18-420b-a2da-ad3a336ff874 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:01.635672 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:01.661093 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 220/444] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:20:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:20:01.673139 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b32225-c87c-4641-8550-d98fb1ac7fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:01.673838 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b32225-c87c-4641-8550-d98fb1ac7fea None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:01.674140 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b32225-c87c-4641-8550-d98fb1ac7fea None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:01.674478 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b32225-c87c-4641-8550-d98fb1ac7fea None admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:01.675372 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b32225-c87c-4641-8550-d98fb1ac7fea None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:01.675917 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b32225-c87c-4641-8550-d98fb1ac7fea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:01.678563 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b32225-c87c-4641-8550-d98fb1ac7fea None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:01.685919 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c83d67d-d722-42d1-8b2b-6fa4892cd1ce None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:01.693084 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 225/445] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jun 7 14:20:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:20:01.700550 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b32225-c87c-4641-8550-d98fb1ac7fea None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:01.807697 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c83d67d-d722-42d1-8b2b-6fa4892cd1ce None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:01.824972 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b32225-c87c-4641-8550-d98fb1ac7fea None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:01.828299 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:01.849628 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81358d92-de57-49d6-9132-41b442bb415c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:01.850406 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81358d92-de57-49d6-9132-41b442bb415c None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:01.850693 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81358d92-de57-49d6-9132-41b442bb415c None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:01.850964 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81358d92-de57-49d6-9132-41b442bb415c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:01.865098 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81358d92-de57-49d6-9132-41b442bb415c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:20:01.870115 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910bc753-5a1c-4f51-bf77-b39ca4517cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:01.870832 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910bc753-5a1c-4f51-bf77-b39ca4517cfa None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:01.871112 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910bc753-5a1c-4f51-bf77-b39ca4517cfa None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:01.871382 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910bc753-5a1c-4f51-bf77-b39ca4517cfa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:01.874890 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-81358d92-de57-49d6-9132-41b442bb415c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:20:01.874890 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:01.874890 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:01.874890 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:01.874890 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:01.874890 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:01.874890 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:01.874890 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:01.874890 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:01.874890 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:01.874890 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:01.874890 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:01.874890 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:20:01.874890 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:20:01.874890 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:20:01.874890 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:20:01.875758 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:20:01.875758 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:20:01.875758 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:20:01.875758 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:20:01.875758 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:01.875758 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:01.875758 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:01.875758 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:01.875758 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:01.875758 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:01.875758 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:01.875758 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:01.875758 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:01.875758 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:01.875758 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:01.875758 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:01.876634 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:01.876634 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:01.876634 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:01.876634 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:01.876634 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:20:01.876634 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:20:01.876634 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:20:01.876634 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:20:01.876634 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:20:01.876634 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:20:01.876634 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:20:01.876634 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:01.879097 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 221/446] 10.222.0.85 () {56 vars in 1096 bytes} [Tue Jun 7 14:20:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:01.886350 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81358d92-de57-49d6-9132-41b442bb415c None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:01.931695 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:01.970941 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910bc753-5a1c-4f51-bf77-b39ca4517cfa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:01.972449 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910bc753-5a1c-4f51-bf77-b39ca4517cfa None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:02.011863 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81358d92-de57-49d6-9132-41b442bb415c None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:02.049974 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b0e247-12c5-4f33-af40-e59d84753f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:02.050652 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b0e247-12c5-4f33-af40-e59d84753f0e None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:02.051222 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b0e247-12c5-4f33-af40-e59d84753f0e None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:02.051222 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b0e247-12c5-4f33-af40-e59d84753f0e None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:02.051968 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b0e247-12c5-4f33-af40-e59d84753f0e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:02.052412 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b0e247-12c5-4f33-af40-e59d84753f0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:02.054207 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b0e247-12c5-4f33-af40-e59d84753f0e None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:02.063500 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 222/447] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jun 7 14:20:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:02.069228 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b0e247-12c5-4f33-af40-e59d84753f0e None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:02.103267 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 226/448] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jun 7 14:20:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 07 14:20:02.109312 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910bc753-5a1c-4f51-bf77-b39ca4517cfa None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:02.122176 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:02.181806 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:02.186189 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b0e247-12c5-4f33-af40-e59d84753f0e None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:02.212501 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910bc753-5a1c-4f51-bf77-b39ca4517cfa None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:02.220245 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fff030-f370-460c-83ae-1b9b1df02ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:02.220924 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fff030-f370-460c-83ae-1b9b1df02ede None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:02.221208 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fff030-f370-460c-83ae-1b9b1df02ede None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:02.221484 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fff030-f370-460c-83ae-1b9b1df02ede None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:02.247173 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1568297a-2a63-4dbb-a3f9-56a03146015a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:02.248095 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1568297a-2a63-4dbb-a3f9-56a03146015a None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:02.248410 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1568297a-2a63-4dbb-a3f9-56a03146015a None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:02.248697 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1568297a-2a63-4dbb-a3f9-56a03146015a None admin] PATH_INFO: `/v3/projects/f95de357ccad493984042656b2c0689a` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:02.249903 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1568297a-2a63-4dbb-a3f9-56a03146015a None admin] RBAC: Authorizing `identity:get_project(project_id=f95de357ccad493984042656b2c0689a)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:02.251359 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1568297a-2a63-4dbb-a3f9-56a03146015a None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:02.253435 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 227/449] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jun 7 14:20:02 2022] GET /identity/v3/projects/f95de357ccad493984042656b2c0689a => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:02.260124 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1568297a-2a63-4dbb-a3f9-56a03146015a None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:02.294852 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fff030-f370-460c-83ae-1b9b1df02ede None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:02.296294 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fff030-f370-460c-83ae-1b9b1df02ede None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:02.334306 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:02.359333 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1568297a-2a63-4dbb-a3f9-56a03146015a None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:02.394004 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0f84ae-9449-4e48-b56e-67756aa95919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:02.394862 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f84ae-9449-4e48-b56e-67756aa95919 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:02.395141 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f84ae-9449-4e48-b56e-67756aa95919 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:02.395413 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f84ae-9449-4e48-b56e-67756aa95919 None admin] PATH_INFO: `/v3/groups/ddcbeccb8288428d8c03e142e4970595` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:02.396616 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0f84ae-9449-4e48-b56e-67756aa95919 None admin] RBAC: Authorizing `identity:get_group(group_id=ddcbeccb8288428d8c03e142e4970595)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:02.396793 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 223/450] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jun 7 14:20:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 14:20:02.398215 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0f84ae-9449-4e48-b56e-67756aa95919 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:02.399837 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 228/451] 10.222.0.85 () {56 vars in 1180 bytes} [Tue Jun 7 14:20:02 2022] GET /identity/v3/groups/ddcbeccb8288428d8c03e142e4970595 => generated 226 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:20:02.402254 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fff030-f370-460c-83ae-1b9b1df02ede None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:02.405117 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0f84ae-9449-4e48-b56e-67756aa95919 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:02.417137 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:02.501585 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fff030-f370-460c-83ae-1b9b1df02ede None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:02.508595 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0f84ae-9449-4e48-b56e-67756aa95919 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:02.535652 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb97dc6-77be-45ad-b705-ef782ca2e289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:02.536540 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb97dc6-77be-45ad-b705-ef782ca2e289 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:02.536832 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb97dc6-77be-45ad-b705-ef782ca2e289 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:02.537112 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb97dc6-77be-45ad-b705-ef782ca2e289 None admin] PATH_INFO: `/v3/projects/61703df7c02543d6ba1f949d9811ee43` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:02.538310 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb97dc6-77be-45ad-b705-ef782ca2e289 None admin] RBAC: Authorizing `identity:get_project(project_id=61703df7c02543d6ba1f949d9811ee43)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:02.539798 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb97dc6-77be-45ad-b705-ef782ca2e289 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:02.541522 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 224/452] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jun 7 14:20:02 2022] GET /identity/v3/projects/61703df7c02543d6ba1f949d9811ee43 => generated 344 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:02.541685 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c2ca3f-89e0-4bd6-a892-33fc5c20a5f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:02.542246 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c2ca3f-89e0-4bd6-a892-33fc5c20a5f7 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:02.542608 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c2ca3f-89e0-4bd6-a892-33fc5c20a5f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:02.542898 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c2ca3f-89e0-4bd6-a892-33fc5c20a5f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:02.543735 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c2ca3f-89e0-4bd6-a892-33fc5c20a5f7 None admin] RBAC: Adding query filter params (group.id=ddcbeccb8288428d8c03e142e4970595, role.id=494cd000d6884a179c334848f52a3a21, scope.project.id=f95de357ccad493984042656b2c0689a) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:02.544051 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c2ca3f-89e0-4bd6-a892-33fc5c20a5f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:02.545513 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c2ca3f-89e0-4bd6-a892-33fc5c20a5f7 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:02.546797 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb97dc6-77be-45ad-b705-ef782ca2e289 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:02.552876 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 229/453] 10.222.0.85 () {56 vars in 1376 bytes} [Tue Jun 7 14:20:02 2022] GET /identity/v3/role_assignments?group.id=ddcbeccb8288428d8c03e142e4970595&scope.project.id=f95de357ccad493984042656b2c0689a&role.id=494cd000d6884a179c334848f52a3a21 => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:02.555811 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:02.656385 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb97dc6-77be-45ad-b705-ef782ca2e289 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:02.700954 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9be0103-4ed3-4999-9489-40ee634cb61d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:02.701833 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9be0103-4ed3-4999-9489-40ee634cb61d None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:02.702115 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9be0103-4ed3-4999-9489-40ee634cb61d None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:02.702415 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9be0103-4ed3-4999-9489-40ee634cb61d None admin] PATH_INFO: `/v3/groups/c1f2253e8d3a4fff9f5f3e3bbc9a247f` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:02.703661 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9be0103-4ed3-4999-9489-40ee634cb61d None admin] RBAC: Authorizing `identity:get_group(group_id=c1f2253e8d3a4fff9f5f3e3bbc9a247f)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:02.705144 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9be0103-4ed3-4999-9489-40ee634cb61d None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:02.706799 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 225/454] 10.222.0.85 () {56 vars in 1180 bytes} [Tue Jun 7 14:20:02 2022] GET /identity/v3/groups/c1f2253e8d3a4fff9f5f3e3bbc9a247f => generated 229 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:20:02.712889 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c2ca3f-89e0-4bd6-a892-33fc5c20a5f7 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:02.791840 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:02.824673 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c2ca3f-89e0-4bd6-a892-33fc5c20a5f7 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:02.860016 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c76d91-8b41-4ed9-8ce1-23ff2fca0d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:02.860730 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c76d91-8b41-4ed9-8ce1-23ff2fca0d46 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:02.861012 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c76d91-8b41-4ed9-8ce1-23ff2fca0d46 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:02.861293 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c76d91-8b41-4ed9-8ce1-23ff2fca0d46 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:02.862133 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c76d91-8b41-4ed9-8ce1-23ff2fca0d46 None admin] RBAC: Adding query filter params (group.id=c1f2253e8d3a4fff9f5f3e3bbc9a247f, role.id=9baff2b18f2649f7b757ff2e1f63c853, scope.project.id=61703df7c02543d6ba1f949d9811ee43) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:02.862447 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c76d91-8b41-4ed9-8ce1-23ff2fca0d46 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:02.863948 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c76d91-8b41-4ed9-8ce1-23ff2fca0d46 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:02.871623 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 230/455] 10.222.0.85 () {56 vars in 1376 bytes} [Tue Jun 7 14:20:02 2022] GET /identity/v3/role_assignments?group.id=c1f2253e8d3a4fff9f5f3e3bbc9a247f&scope.project.id=61703df7c02543d6ba1f949d9811ee43&role.id=9baff2b18f2649f7b757ff2e1f63c853 => generated 611 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:05.881285 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab56ba9-bced-4ec2-8223-8e476c32c3a2 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:05.881808 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab56ba9-bced-4ec2-8223-8e476c32c3a2 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:05.881808 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab56ba9-bced-4ec2-8223-8e476c32c3a2 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:05.883450 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 226/456] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:20:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:20:05.897173 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7960c931-9453-4f6c-8078-b4bfc8fcde38 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:05.897677 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7960c931-9453-4f6c-8078-b4bfc8fcde38 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:05.897750 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7960c931-9453-4f6c-8078-b4bfc8fcde38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:05.921672 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-7960c931-9453-4f6c-8078-b4bfc8fcde38 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:20:05.923248 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-7960c931-9453-4f6c-8078-b4bfc8fcde38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:20:05.980348 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6174774a-5f0f-4c18-8d5d-03f6df4c24be None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:05.980348 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6174774a-5f0f-4c18-8d5d-03f6df4c24be None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:05.980348 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6174774a-5f0f-4c18-8d5d-03f6df4c24be None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:05.980348 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 227/457] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:20:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 14:20:05.985890 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0408008c-e79f-4547-89df-6928a7b3b7c3 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:05.986172 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0408008c-e79f-4547-89df-6928a7b3b7c3 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:05.986448 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0408008c-e79f-4547-89df-6928a7b3b7c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:06.015929 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-0408008c-e79f-4547-89df-6928a7b3b7c3 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:20:06.017331 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-0408008c-e79f-4547-89df-6928a7b3b7c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:20:06.034644 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 231/458] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:20:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:20:06.036950 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09e4740-bbed-459a-8aea-b8297d40bc17 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:06.037255 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09e4740-bbed-459a-8aea-b8297d40bc17 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:06.037542 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09e4740-bbed-459a-8aea-b8297d40bc17 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:06.039889 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 232/459] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:20:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:20:06.045734 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab128185-a2a4-495d-b12a-9ce1866b9a82 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:06.046022 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab128185-a2a4-495d-b12a-9ce1866b9a82 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:06.046300 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab128185-a2a4-495d-b12a-9ce1866b9a82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:06.069613 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-ab128185-a2a4-495d-b12a-9ce1866b9a82 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:20:06.071531 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-ab128185-a2a4-495d-b12a-9ce1866b9a82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:20:06.105376 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 228/460] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:20:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 14:20:06.106636 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0408008c-e79f-4547-89df-6928a7b3b7c3 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:06.110916 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:06.156887 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:06.176397 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 233/461] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:20:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:20:06.176397 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab128185-a2a4-495d-b12a-9ce1866b9a82 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:06.253457 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0408008c-e79f-4547-89df-6928a7b3b7c3 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:06.284705 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab128185-a2a4-495d-b12a-9ce1866b9a82 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:06.302100 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4cba10-fb27-4a37-b4a4-a521c582726d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:06.302909 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4cba10-fb27-4a37-b4a4-a521c582726d None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:06.303227 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4cba10-fb27-4a37-b4a4-a521c582726d None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:06.303744 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4cba10-fb27-4a37-b4a4-a521c582726d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:06.317974 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4cba10-fb27-4a37-b4a4-a521c582726d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:06.320125 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4cba10-fb27-4a37-b4a4-a521c582726d None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:06.325134 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d2679b-2aea-45d9-8846-05dfa3ac2fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:06.326326 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d2679b-2aea-45d9-8846-05dfa3ac2fc0 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:06.326870 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d2679b-2aea-45d9-8846-05dfa3ac2fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:06.327348 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d2679b-2aea-45d9-8846-05dfa3ac2fc0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:06.339421 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d2679b-2aea-45d9-8846-05dfa3ac2fc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:06.341381 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d2679b-2aea-45d9-8846-05dfa3ac2fc0 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:06.341757 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-fa4cba10-fb27-4a37-b4a4-a521c582726d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:06.341757 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:06.341757 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:06.341757 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:06.341757 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:06.341757 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:06.341757 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:06.341757 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:06.341757 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:06.341757 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:06.341757 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:06.341757 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:06.341757 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:20:06.341757 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:20:06.341757 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:20:06.342721 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:20:06.342721 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:06.342721 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:06.342721 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:06.342721 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:06.342721 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:06.342721 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:06.342721 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:06.342721 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:06.342721 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:06.342721 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:06.342721 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:06.342721 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:06.342721 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:06.342721 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:06.342721 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:06.342721 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:06.343561 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:20:06.343561 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:20:06.343561 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:06.343561 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:06.343561 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 229/462] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:20:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:06.344266 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4cba10-fb27-4a37-b4a4-a521c582726d None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:06.364490 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-23d2679b-2aea-45d9-8846-05dfa3ac2fc0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:06.364490 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:06.364490 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:06.364490 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:06.364490 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:06.364490 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:06.364490 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:06.364490 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:06.364490 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:06.364490 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:06.364490 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:06.364490 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:06.364490 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:20:06.364490 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:20:06.364490 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:20:06.365407 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:20:06.365407 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:06.365407 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:06.365407 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:06.365407 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:06.365407 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:06.365407 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:06.365407 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:06.365407 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:06.365407 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:06.365407 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:06.365407 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:06.365407 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:06.365407 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:06.365407 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:06.365407 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:06.365407 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:06.366229 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:20:06.366229 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:20:06.366229 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:06.366229 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:06.366229 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 234/463] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:20:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:06.367071 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d2679b-2aea-45d9-8846-05dfa3ac2fc0 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:06.418489 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:06.420313 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:06.465220 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4cba10-fb27-4a37-b4a4-a521c582726d None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:06.474018 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d2679b-2aea-45d9-8846-05dfa3ac2fc0 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:06.503871 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5474b5-89ef-4244-b01e-9c6d67dc1bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:06.504631 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5474b5-89ef-4244-b01e-9c6d67dc1bc5 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:06.504919 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5474b5-89ef-4244-b01e-9c6d67dc1bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:06.505203 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5474b5-89ef-4244-b01e-9c6d67dc1bc5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:06.512969 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6951c95-bce7-41f8-bf5f-6e788200928e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:06.513535 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6951c95-bce7-41f8-bf5f-6e788200928e None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:06.513825 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6951c95-bce7-41f8-bf5f-6e788200928e None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:06.514104 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6951c95-bce7-41f8-bf5f-6e788200928e None admin] PATH_INFO: `/v3/domains` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:06.514882 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6951c95-bce7-41f8-bf5f-6e788200928e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:06.515145 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6951c95-bce7-41f8-bf5f-6e788200928e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:06.516554 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6951c95-bce7-41f8-bf5f-6e788200928e None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:06.517269 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5474b5-89ef-4244-b01e-9c6d67dc1bc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:06.518714 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5474b5-89ef-4244-b01e-9c6d67dc1bc5 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:06.539232 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 235/464] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jun 7 14:20:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:06.539232 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0dfb2a-d756-44ec-9107-908c7063110a None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:06.539232 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0dfb2a-d756-44ec-9107-908c7063110a None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:06.539232 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0dfb2a-d756-44ec-9107-908c7063110a None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:06.539682 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-ac5474b5-89ef-4244-b01e-9c6d67dc1bc5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:06.539682 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:06.539682 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:06.539682 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:06.539682 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:06.539682 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:06.539682 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:06.539682 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:06.539682 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:06.539682 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:06.539682 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:06.539682 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:06.539682 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:20:06.539682 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:20:06.539682 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:20:06.540635 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:20:06.540635 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:06.540635 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:06.540635 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:06.540635 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:06.540635 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:06.540635 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:06.540635 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:06.540635 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:06.540635 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:06.540635 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:06.540635 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:06.540635 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:06.540635 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:06.540635 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:06.540635 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:06.540635 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:06.541693 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:20:06.541693 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:20:06.541693 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:06.541693 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:06.541945 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 236/465] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:20:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:20:06.541945 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0dfb2a-d756-44ec-9107-908c7063110a None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:06.542094 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 230/466] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:20:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:06.548124 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5474b5-89ef-4244-b01e-9c6d67dc1bc5 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:06.664129 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5474b5-89ef-4244-b01e-9c6d67dc1bc5 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:06.671106 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0dfb2a-d756-44ec-9107-908c7063110a None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:06.674989 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:06.680818 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:06.704998 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e577a7e-b393-4393-8872-4098d418ef21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:06.704998 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e577a7e-b393-4393-8872-4098d418ef21 None admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:06.704998 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e577a7e-b393-4393-8872-4098d418ef21 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:06.705327 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e577a7e-b393-4393-8872-4098d418ef21 None admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:06.706028 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e577a7e-b393-4393-8872-4098d418ef21 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:06.707518 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e577a7e-b393-4393-8872-4098d418ef21 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:06.709382 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212f7193-77ff-4af5-95c9-a4f97e652291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:06.709382 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212f7193-77ff-4af5-95c9-a4f97e652291 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:06.709983 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212f7193-77ff-4af5-95c9-a4f97e652291 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:06.710263 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212f7193-77ff-4af5-95c9-a4f97e652291 None admin] PATH_INFO: `/v3/domains` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:06.711824 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212f7193-77ff-4af5-95c9-a4f97e652291 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:06.712138 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212f7193-77ff-4af5-95c9-a4f97e652291 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:06.713542 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212f7193-77ff-4af5-95c9-a4f97e652291 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:06.725489 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 237/467] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jun 7 14:20:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:06.732188 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26db3c0d-51f0-42c2-b5b0-3575b2ad96ab None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:06.732188 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26db3c0d-51f0-42c2-b5b0-3575b2ad96ab None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:06.732188 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26db3c0d-51f0-42c2-b5b0-3575b2ad96ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:06.744144 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-26db3c0d-51f0-42c2-b5b0-3575b2ad96ab None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:20:06.745801 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-26db3c0d-51f0-42c2-b5b0-3575b2ad96ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:20:06.827121 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 238/468] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:20:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:20:06.827970 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26db3c0d-51f0-42c2-b5b0-3575b2ad96ab None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:06.855491 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 231/469] 10.222.0.85 () {60 vars in 1132 bytes} [Tue Jun 7 14:20:06 2022] POST /identity/v3/users => generated 246 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:20:06.856615 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e577a7e-b393-4393-8872-4098d418ef21 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:06.945558 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:06.950229 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26db3c0d-51f0-42c2-b5b0-3575b2ad96ab None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:07.002522 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8447cd3c-83e4-45bd-a843-1ef740cbe1cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:07.003697 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8447cd3c-83e4-45bd-a843-1ef740cbe1cd None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:07.004121 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8447cd3c-83e4-45bd-a843-1ef740cbe1cd None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:07.004551 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8447cd3c-83e4-45bd-a843-1ef740cbe1cd None admin] PATH_INFO: `/v3/domains` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:07.005439 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8447cd3c-83e4-45bd-a843-1ef740cbe1cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:07.005864 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8447cd3c-83e4-45bd-a843-1ef740cbe1cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:07.006289 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e577a7e-b393-4393-8872-4098d418ef21 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:07.007564 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8447cd3c-83e4-45bd-a843-1ef740cbe1cd None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:07.028541 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 239/470] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jun 7 14:20:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:07.029764 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8447cd3c-83e4-45bd-a843-1ef740cbe1cd None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:07.057603 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fc63ae-ad24-41d2-8066-a44ef81fb6a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:07.058784 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fc63ae-ad24-41d2-8066-a44ef81fb6a3 None admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:07.059300 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fc63ae-ad24-41d2-8066-a44ef81fb6a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:07.059842 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fc63ae-ad24-41d2-8066-a44ef81fb6a3 None admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:07.060953 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fc63ae-ad24-41d2-8066-a44ef81fb6a3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:07.062767 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fc63ae-ad24-41d2-8066-a44ef81fb6a3 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:07.148600 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 232/471] 10.222.0.85 () {60 vars in 1132 bytes} [Tue Jun 7 14:20:06 2022] POST /identity/v3/users => generated 248 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:20:07.149256 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fc63ae-ad24-41d2-8066-a44ef81fb6a3 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:07.150533 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8447cd3c-83e4-45bd-a843-1ef740cbe1cd None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:07.178126 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:07.205640 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebcfa47-37dd-4f4f-a257-308ba1ad5dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:07.206580 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebcfa47-37dd-4f4f-a257-308ba1ad5dd0 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:07.206870 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebcfa47-37dd-4f4f-a257-308ba1ad5dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:07.207404 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebcfa47-37dd-4f4f-a257-308ba1ad5dd0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:07.224603 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebcfa47-37dd-4f4f-a257-308ba1ad5dd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:07.227592 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebcfa47-37dd-4f4f-a257-308ba1ad5dd0 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:07.253920 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-cebcfa47-37dd-4f4f-a257-308ba1ad5dd0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:07.253920 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:07.253920 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:07.253920 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:07.253920 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:07.253920 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:07.253920 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:07.253920 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:07.253920 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:07.253920 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:07.253920 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:07.253920 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:07.253920 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:20:07.253920 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:20:07.253920 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:20:07.254812 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:20:07.254812 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:07.254812 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:07.254812 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:07.254812 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:07.254812 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:07.254812 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:07.254812 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:07.254812 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:07.254812 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:07.254812 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:07.254812 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:07.254812 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:07.254812 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:07.254812 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:07.254812 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:07.254812 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:07.255643 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:20:07.255643 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:20:07.255643 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:07.255643 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:07.255643 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 240/472] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:20:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:07.256922 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de576afd-31f5-41f4-b6b5-1b7e254306a3 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:07.257719 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de576afd-31f5-41f4-b6b5-1b7e254306a3 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:07.258002 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de576afd-31f5-41f4-b6b5-1b7e254306a3 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:07.259958 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 241/473] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:20:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:20:07.265409 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de576afd-31f5-41f4-b6b5-1b7e254306a3 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:07.280340 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fc63ae-ad24-41d2-8066-a44ef81fb6a3 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:07.285643 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:07.321720 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d308a197-1991-4cff-9887-734c04800aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:07.322519 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d308a197-1991-4cff-9887-734c04800aac None admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:07.322808 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d308a197-1991-4cff-9887-734c04800aac None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:07.323086 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d308a197-1991-4cff-9887-734c04800aac None admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:07.323808 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d308a197-1991-4cff-9887-734c04800aac None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:07.325343 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d308a197-1991-4cff-9887-734c04800aac None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:07.398917 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de576afd-31f5-41f4-b6b5-1b7e254306a3 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:07.436880 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76f2cef-bf0e-4d1b-aa5d-eb7ac0ffed77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:07.437944 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76f2cef-bf0e-4d1b-aa5d-eb7ac0ffed77 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:07.438434 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76f2cef-bf0e-4d1b-aa5d-eb7ac0ffed77 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:07.438537 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 233/474] 10.222.0.85 () {60 vars in 1132 bytes} [Tue Jun 7 14:20:07 2022] POST /identity/v3/users => generated 248 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:20:07.439022 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76f2cef-bf0e-4d1b-aa5d-eb7ac0ffed77 None admin] PATH_INFO: `/v3/domains` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:07.439841 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20c0ec4-54a8-4b5b-a133-8ac28d76fa38 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:07.440138 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20c0ec4-54a8-4b5b-a133-8ac28d76fa38 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:07.440465 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20c0ec4-54a8-4b5b-a133-8ac28d76fa38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:07.440707 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76f2cef-bf0e-4d1b-aa5d-eb7ac0ffed77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:07.441208 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76f2cef-bf0e-4d1b-aa5d-eb7ac0ffed77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:07.442866 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76f2cef-bf0e-4d1b-aa5d-eb7ac0ffed77 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:07.462637 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 242/475] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jun 7 14:20:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:07.467590 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-a20c0ec4-54a8-4b5b-a133-8ac28d76fa38 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:20:07.468295 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76f2cef-bf0e-4d1b-aa5d-eb7ac0ffed77 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:07.469305 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-a20c0ec4-54a8-4b5b-a133-8ac28d76fa38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:20:07.562253 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 234/476] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:20:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:20:07.597521 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:07.601885 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20c0ec4-54a8-4b5b-a133-8ac28d76fa38 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:07.615163 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:07.623243 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76f2cef-bf0e-4d1b-aa5d-eb7ac0ffed77 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:07.667935 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c7090c-eb49-49df-9e4d-0c350b2a8964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:07.669509 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c7090c-eb49-49df-9e4d-0c350b2a8964 None admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:07.670065 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c7090c-eb49-49df-9e4d-0c350b2a8964 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:07.670643 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c7090c-eb49-49df-9e4d-0c350b2a8964 None admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:07.671770 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c7090c-eb49-49df-9e4d-0c350b2a8964 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:07.674022 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c7090c-eb49-49df-9e4d-0c350b2a8964 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:07.766595 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20c0ec4-54a8-4b5b-a133-8ac28d76fa38 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:07.785878 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 243/477] 10.222.0.85 () {60 vars in 1132 bytes} [Tue Jun 7 14:20:07 2022] POST /identity/v3/users => generated 249 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:20:07.814470 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960ec800-3af9-46f2-8d91-79dcc31a2f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:07.815355 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ec800-3af9-46f2-8d91-79dcc31a2f36 None admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:07.815743 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ec800-3af9-46f2-8d91-79dcc31a2f36 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:07.816123 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ec800-3af9-46f2-8d91-79dcc31a2f36 None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:07.817085 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960ec800-3af9-46f2-8d91-79dcc31a2f36 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:07.818714 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960ec800-3af9-46f2-8d91-79dcc31a2f36 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:07.864112 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.sql.core [None req-960ec800-3af9-46f2-8d91-79dcc31a2f36 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 07 14:20:07.864112 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 07 14:20:07.864112 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [parameters: {'id': '4f8eaeae2b404dd5b45e5f525e9661a2', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 07 14:20:07.864112 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80388) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 07 14:20:07.883355 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-960ec800-3af9-46f2-8d91-79dcc31a2f36 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 07 14:20:07.883355 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:07.883355 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 07 14:20:07.883355 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 07 14:20:07.883355 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 07 14:20:07.883355 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 07 14:20:07.883355 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 07 14:20:07.883355 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application result = self._query(query) Jun 07 14:20:07.883355 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 07 14:20:07.883355 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application conn.query(q) Jun 07 14:20:07.883355 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 07 14:20:07.883355 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 07 14:20:07.883355 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 07 14:20:07.883355 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application result.read() Jun 07 14:20:07.883355 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 07 14:20:07.884278 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 07 14:20:07.884278 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 07 14:20:07.884278 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 07 14:20:07.884278 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 07 14:20:07.884278 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 07 14:20:07.884278 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 07 14:20:07.884278 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 07 14:20:07.884278 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 07 14:20:07.884278 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:07.884278 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 07 14:20:07.884278 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:07.884278 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:07.884278 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 07 14:20:07.884278 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 07 14:20:07.884278 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 07 14:20:07.884278 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return ref.to_dict() Jun 07 14:20:07.884278 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 07 14:20:07.884278 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application next(self.gen) Jun 07 14:20:07.885249 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 07 14:20:07.885249 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application yield resource Jun 07 14:20:07.885249 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 07 14:20:07.885249 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application next(self.gen) Jun 07 14:20:07.885249 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 07 14:20:07.885249 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application self.session.rollback() Jun 07 14:20:07.885249 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 07 14:20:07.885249 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application self.force_reraise() Jun 07 14:20:07.885249 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 07 14:20:07.885249 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise self.value Jun 07 14:20:07.885249 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 07 14:20:07.885249 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 07 14:20:07.885249 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 07 14:20:07.885249 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application session.commit() Jun 07 14:20:07.885249 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 07 14:20:07.885249 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 07 14:20:07.885249 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 07 14:20:07.887249 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application self._prepare_impl() Jun 07 14:20:07.887249 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 07 14:20:07.887249 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application self.session.flush() Jun 07 14:20:07.887249 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 07 14:20:07.887249 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application self._flush(objects) Jun 07 14:20:07.887249 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 07 14:20:07.887249 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 07 14:20:07.887249 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 07 14:20:07.887249 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application compat.raise_( Jun 07 14:20:07.887249 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 07 14:20:07.887249 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception Jun 07 14:20:07.887249 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 07 14:20:07.887249 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application flush_context.execute() Jun 07 14:20:07.887249 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 07 14:20:07.887249 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rec.execute(self) Jun 07 14:20:07.887249 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 07 14:20:07.887249 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 07 14:20:07.889503 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 07 14:20:07.889503 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 07 14:20:07.889503 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 07 14:20:07.889503 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 07 14:20:07.889503 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 07 14:20:07.889503 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 07 14:20:07.889503 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 07 14:20:07.889503 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 07 14:20:07.889503 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 07 14:20:07.889503 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 07 14:20:07.889503 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 07 14:20:07.889503 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 07 14:20:07.889503 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 07 14:20:07.889503 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 07 14:20:07.889503 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 07 14:20:07.889503 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception Jun 07 14:20:07.891779 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 07 14:20:07.891779 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 07 14:20:07.891779 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 07 14:20:07.891779 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 07 14:20:07.891779 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 07 14:20:07.891779 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application result = self._query(query) Jun 07 14:20:07.891779 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 07 14:20:07.891779 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application conn.query(q) Jun 07 14:20:07.891779 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 07 14:20:07.891779 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 07 14:20:07.891779 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 07 14:20:07.891779 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application result.read() Jun 07 14:20:07.891779 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 07 14:20:07.891779 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 07 14:20:07.891779 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 07 14:20:07.891779 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 07 14:20:07.893871 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 07 14:20:07.893871 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 07 14:20:07.893871 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 07 14:20:07.893871 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 07 14:20:07.893871 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 07 14:20:07.893871 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 07 14:20:07.893871 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [parameters: {'id': '4f8eaeae2b404dd5b45e5f525e9661a2', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 07 14:20:07.893871 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 07 14:20:07.893871 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:07.893871 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 14:20:07.893871 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:07.893871 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:07.893871 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:07.893871 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:07.893871 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:07.893871 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:07.898530 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:07.898530 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:07.898530 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:07.898530 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:07.898530 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:07.898530 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:07.898530 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 07 14:20:07.898530 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 07 14:20:07.898530 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:07.898530 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:07.898530 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 07 14:20:07.898530 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 07 14:20:07.898530 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 07 14:20:07.898530 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 07 14:20:07.898530 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 07 14:20:07.898530 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:07.901423 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 235/478] 10.222.0.85 () {60 vars in 1131 bytes} [Tue Jun 7 14:20:07 2022] POST /identity/v3/roles => generated 144 bytes in 284 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:07.901591 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c7090c-eb49-49df-9e4d-0c350b2a8964 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:08.044339 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c7090c-eb49-49df-9e4d-0c350b2a8964 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:08.072715 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:08.088738 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b05c962-88a8-408b-9ba8-2a8f52ea9d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:08.089122 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b05c962-88a8-408b-9ba8-2a8f52ea9d00 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:08.091645 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b05c962-88a8-408b-9ba8-2a8f52ea9d00 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:08.091645 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b05c962-88a8-408b-9ba8-2a8f52ea9d00 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:08.101971 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b05c962-88a8-408b-9ba8-2a8f52ea9d00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:20:08.113950 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-6b05c962-88a8-408b-9ba8-2a8f52ea9d00 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 14:20:08.113950 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:08.113950 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:08.113950 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:08.113950 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:08.113950 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:08.113950 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:08.113950 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:08.113950 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:08.113950 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:08.113950 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:08.113950 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:08.113950 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:20:08.113950 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:20:08.113950 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:20:08.113950 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:20:08.114859 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:20:08.114859 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:20:08.114859 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:20:08.114859 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:20:08.114859 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:08.114859 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:08.114859 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:08.114859 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:08.114859 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:08.114859 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:08.114859 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:08.114859 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:08.114859 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:08.114859 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:08.114859 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:08.114859 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:08.115645 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:08.115645 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:08.115645 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:08.115645 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:08.115645 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:20:08.115645 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:20:08.115645 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:20:08.115645 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:20:08.115645 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:20:08.115645 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:20:08.115645 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 14:20:08.115645 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:08.121005 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 244/479] 10.222.0.85 () {56 vars in 1114 bytes} [Tue Jun 7 14:20:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:08.126580 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960ec800-3af9-46f2-8d91-79dcc31a2f36 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:08.212777 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:08.246460 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960ec800-3af9-46f2-8d91-79dcc31a2f36 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:08.283392 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea4b4e7-e4bc-41d2-8c7e-cb26fbf1a210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:08.284207 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea4b4e7-e4bc-41d2-8c7e-cb26fbf1a210 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:08.284584 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea4b4e7-e4bc-41d2-8c7e-cb26fbf1a210 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:08.284894 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea4b4e7-e4bc-41d2-8c7e-cb26fbf1a210 None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:08.285674 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea4b4e7-e4bc-41d2-8c7e-cb26fbf1a210 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:08.285986 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea4b4e7-e4bc-41d2-8c7e-cb26fbf1a210 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:08.287425 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea4b4e7-e4bc-41d2-8c7e-cb26fbf1a210 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:08.297444 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 236/480] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jun 7 14:20:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:09.616435 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c360cfe-bacc-49a7-9d68-21d7df12f73e None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:09.617521 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c360cfe-bacc-49a7-9d68-21d7df12f73e None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:09.617595 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c360cfe-bacc-49a7-9d68-21d7df12f73e None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:09.619202 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 245/481] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:20:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:20:09.625240 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adcc1a0-1552-4c74-be07-b6dfe1ad694c None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:09.625674 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adcc1a0-1552-4c74-be07-b6dfe1ad694c None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:09.625853 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adcc1a0-1552-4c74-be07-b6dfe1ad694c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:09.645481 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-3adcc1a0-1552-4c74-be07-b6dfe1ad694c None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:20:09.646709 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-3adcc1a0-1552-4c74-be07-b6dfe1ad694c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:20:09.730063 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 237/482] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:20:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:20:09.757227 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c360cfe-bacc-49a7-9d68-21d7df12f73e None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:09.910153 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c360cfe-bacc-49a7-9d68-21d7df12f73e None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:09.955746 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df22025d-6ad8-4a38-bb29-4abe7e65de1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:09.956576 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df22025d-6ad8-4a38-bb29-4abe7e65de1f None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:09.956866 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df22025d-6ad8-4a38-bb29-4abe7e65de1f None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:09.957149 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df22025d-6ad8-4a38-bb29-4abe7e65de1f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:09.969165 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df22025d-6ad8-4a38-bb29-4abe7e65de1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:20:09.977579 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-df22025d-6ad8-4a38-bb29-4abe7e65de1f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:20:09.977579 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:09.977579 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:09.977579 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:09.977579 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:09.977579 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:09.977579 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:09.977579 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:09.977579 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:09.977579 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:09.977579 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:09.977579 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:09.977579 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:20:09.977579 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:20:09.977579 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:20:09.977579 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:20:09.978454 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:20:09.978454 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:20:09.978454 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:20:09.978454 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:20:09.978454 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:09.978454 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:09.978454 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:09.978454 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:09.978454 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:09.978454 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:09.978454 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:09.978454 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:09.978454 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:09.978454 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:09.978454 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:09.978454 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:09.979253 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:09.979253 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:09.979253 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:09.979253 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:09.979253 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:20:09.979253 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:20:09.979253 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:20:09.979253 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:20:09.979253 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:20:09.979253 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:20:09.979253 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:20:09.979253 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:09.979253 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 246/483] 10.222.0.85 () {56 vars in 1102 bytes} [Tue Jun 7 14:20:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:09.984946 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adcc1a0-1552-4c74-be07-b6dfe1ad694c None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:10.079743 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f02a283-c945-4d5e-865a-8299d6984f94 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:10.080097 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f02a283-c945-4d5e-865a-8299d6984f94 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:10.080326 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f02a283-c945-4d5e-865a-8299d6984f94 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:10.081911 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 247/484] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:20:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 14:20:10.089418 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df05a4a-04b7-4154-a076-6b9b061ea940 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:10.090033 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df05a4a-04b7-4154-a076-6b9b061ea940 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:10.090574 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df05a4a-04b7-4154-a076-6b9b061ea940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:10.105989 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:10.113516 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-0df05a4a-04b7-4154-a076-6b9b061ea940 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:20:10.114994 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-0df05a4a-04b7-4154-a076-6b9b061ea940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:20:10.170151 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:10.188682 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adcc1a0-1552-4c74-be07-b6dfe1ad694c None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:10.215167 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 248/485] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:20:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 14:20:10.240312 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73723cf7-3fbb-4a12-aecf-80d1bce465be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:10.241413 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73723cf7-3fbb-4a12-aecf-80d1bce465be None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:10.241920 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73723cf7-3fbb-4a12-aecf-80d1bce465be None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:10.242423 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73723cf7-3fbb-4a12-aecf-80d1bce465be None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:10.243380 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73723cf7-3fbb-4a12-aecf-80d1bce465be None admin] RBAC: Adding query filter params (name=service) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:10.243926 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73723cf7-3fbb-4a12-aecf-80d1bce465be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:10.245764 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73723cf7-3fbb-4a12-aecf-80d1bce465be None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:10.254957 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 238/486] 10.222.0.85 () {56 vars in 1103 bytes} [Tue Jun 7 14:20:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:10.260984 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df05a4a-04b7-4154-a076-6b9b061ea940 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:10.271484 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73723cf7-3fbb-4a12-aecf-80d1bce465be None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:10.413895 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df05a4a-04b7-4154-a076-6b9b061ea940 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:10.438210 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73723cf7-3fbb-4a12-aecf-80d1bce465be None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:10.449229 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ffaa68-81f1-406b-914c-afcb1a81b0a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:10.450062 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ffaa68-81f1-406b-914c-afcb1a81b0a5 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:10.450350 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ffaa68-81f1-406b-914c-afcb1a81b0a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:10.450631 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ffaa68-81f1-406b-914c-afcb1a81b0a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:10.463052 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ffaa68-81f1-406b-914c-afcb1a81b0a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:10.464701 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ffaa68-81f1-406b-914c-afcb1a81b0a5 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:10.469046 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:10.481948 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe43ca7-09c8-4fe3-86da-28e7f97a3122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:10.483193 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe43ca7-09c8-4fe3-86da-28e7f97a3122 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:10.483667 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe43ca7-09c8-4fe3-86da-28e7f97a3122 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:10.484103 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe43ca7-09c8-4fe3-86da-28e7f97a3122 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:10.490213 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-f5ffaa68-81f1-406b-914c-afcb1a81b0a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:10.490213 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:10.490213 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:10.490213 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:10.490213 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:10.490213 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:10.490213 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:10.490213 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:10.490213 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:10.490213 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:10.490213 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:10.490213 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:10.490213 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:20:10.490213 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:20:10.490213 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:20:10.491066 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:20:10.491066 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:10.491066 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:10.491066 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:10.491066 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:10.491066 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:10.491066 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:10.491066 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:10.491066 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:10.491066 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:10.491066 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:10.491066 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:10.491066 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:10.491066 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:10.491066 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:10.491066 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:10.491066 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:10.492333 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:20:10.492333 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:20:10.492333 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:10.492333 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:10.492333 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 249/487] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:20:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:10.497422 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c353b060-af9c-4498-9568-9134f6ed0476 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:10.497714 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c353b060-af9c-4498-9568-9134f6ed0476 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:10.498057 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c353b060-af9c-4498-9568-9134f6ed0476 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:10.499515 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 250/488] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:20:10 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:20:10.503638 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c353b060-af9c-4498-9568-9134f6ed0476 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:10.508501 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe43ca7-09c8-4fe3-86da-28e7f97a3122 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:20:10.516354 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-dbe43ca7-09c8-4fe3-86da-28e7f97a3122 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:20:10.516354 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:10.516354 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:10.516354 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:10.516354 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:10.516354 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:10.516354 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:10.516354 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:10.516354 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:10.516354 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:10.516354 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:10.516354 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:10.516354 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:20:10.516354 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:20:10.516354 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:20:10.516354 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:20:10.517393 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:20:10.517393 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:20:10.517393 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:20:10.517393 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:20:10.517393 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:10.517393 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:10.517393 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:10.517393 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:10.517393 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:10.517393 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:10.517393 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:10.517393 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:10.517393 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:10.517393 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:10.517393 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:10.517393 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:10.518170 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:10.518170 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:10.518170 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:10.518170 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:10.518170 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:20:10.518170 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:20:10.518170 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:20:10.518170 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:20:10.518170 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:20:10.518170 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:20:10.518170 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:20:10.518170 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:10.522412 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 239/489] 10.222.0.85 () {56 vars in 1102 bytes} [Tue Jun 7 14:20:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:10.524410 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f0cc22-3525-4ed4-8572-714a46b4a208 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:10.524701 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f0cc22-3525-4ed4-8572-714a46b4a208 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:10.524977 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f0cc22-3525-4ed4-8572-714a46b4a208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:10.540810 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:10.543409 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-77f0cc22-3525-4ed4-8572-714a46b4a208 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:20:10.544642 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-77f0cc22-3525-4ed4-8572-714a46b4a208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:20:10.628472 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 240/490] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:20:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:20:10.628472 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f0cc22-3525-4ed4-8572-714a46b4a208 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:10.658394 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c353b060-af9c-4498-9568-9134f6ed0476 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:10.703175 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f70385-e657-41c1-9af2-48081c1e591c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:10.703949 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f70385-e657-41c1-9af2-48081c1e591c None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:10.713592 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f70385-e657-41c1-9af2-48081c1e591c None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:10.713592 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f70385-e657-41c1-9af2-48081c1e591c None admin] PATH_INFO: `/v3/domains` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:10.713592 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f70385-e657-41c1-9af2-48081c1e591c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:10.713592 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f70385-e657-41c1-9af2-48081c1e591c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:10.713592 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f70385-e657-41c1-9af2-48081c1e591c None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:10.721003 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 251/491] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jun 7 14:20:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:10.722204 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f70385-e657-41c1-9af2-48081c1e591c None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:10.771412 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f0cc22-3525-4ed4-8572-714a46b4a208 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:10.790152 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:10.825292 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc0a184-93a7-446d-bc9c-16cead85355d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:10.826393 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc0a184-93a7-446d-bc9c-16cead85355d None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:10.826990 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc0a184-93a7-446d-bc9c-16cead85355d None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:10.827343 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc0a184-93a7-446d-bc9c-16cead85355d None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:10.828202 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc0a184-93a7-446d-bc9c-16cead85355d None admin] RBAC: Adding query filter params (name=service) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:10.828574 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc0a184-93a7-446d-bc9c-16cead85355d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:10.830150 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc0a184-93a7-446d-bc9c-16cead85355d None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:10.839601 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 241/492] 10.222.0.85 () {56 vars in 1103 bytes} [Tue Jun 7 14:20:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:10.840364 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc0a184-93a7-446d-bc9c-16cead85355d None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:10.853177 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:10.865447 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f70385-e657-41c1-9af2-48081c1e591c None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:10.903375 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c616aa0-f10c-4019-b041-2dc660b894f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:10.904175 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c616aa0-f10c-4019-b041-2dc660b894f7 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:10.904483 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c616aa0-f10c-4019-b041-2dc660b894f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:10.904764 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c616aa0-f10c-4019-b041-2dc660b894f7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:10.915350 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c616aa0-f10c-4019-b041-2dc660b894f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:20:10.923131 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-4c616aa0-f10c-4019-b041-2dc660b894f7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:20:10.923131 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:10.923131 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:10.923131 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:10.923131 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:10.923131 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:10.923131 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:10.923131 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:10.923131 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:10.923131 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:10.923131 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:10.923131 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:10.923131 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:20:10.923131 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:20:10.923131 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:20:10.923131 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:20:10.924008 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:20:10.924008 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:20:10.924008 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:20:10.924008 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:20:10.924008 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:10.924008 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:10.924008 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:10.924008 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:10.924008 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:10.924008 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:10.924008 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:10.924008 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:10.924008 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:10.924008 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:10.924008 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:10.924008 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:10.925257 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:10.925257 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:10.925257 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:10.925257 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:10.925257 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:20:10.925257 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:20:10.925257 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:20:10.925257 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:20:10.925257 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:20:10.925257 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:20:10.925257 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:20:10.925257 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:10.925257 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 252/493] 10.222.0.85 () {56 vars in 1102 bytes} [Tue Jun 7 14:20:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:10.930876 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73675b5e-1417-460e-9776-9be3c1180b85 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:10.931175 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73675b5e-1417-460e-9776-9be3c1180b85 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:10.931454 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73675b5e-1417-460e-9776-9be3c1180b85 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:10.933043 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 253/494] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:20:10 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:20:10.933519 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73675b5e-1417-460e-9776-9be3c1180b85 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:10.964646 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc0a184-93a7-446d-bc9c-16cead85355d None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:11.011470 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71abba31-33f2-4ff8-873e-101d97f91f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:11.012258 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71abba31-33f2-4ff8-873e-101d97f91f1a None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:11.012577 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71abba31-33f2-4ff8-873e-101d97f91f1a None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:11.012888 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71abba31-33f2-4ff8-873e-101d97f91f1a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:11.022914 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71abba31-33f2-4ff8-873e-101d97f91f1a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:11.024843 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71abba31-33f2-4ff8-873e-101d97f91f1a None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:11.041927 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-71abba31-33f2-4ff8-873e-101d97f91f1a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 07 14:20:11.041927 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:11.041927 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:11.041927 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:11.041927 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:11.041927 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:11.041927 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:11.041927 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:11.041927 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:11.041927 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:11.041927 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:11.041927 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:11.041927 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 14:20:11.041927 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 14:20:11.041927 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 14:20:11.041927 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 14:20:11.042922 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:11.042922 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:11.042922 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 14:20:11.042922 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:20:11.042922 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 14:20:11.042922 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:20:11.042922 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:11.042922 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:11.042922 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:11.042922 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:11.042922 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:11.042922 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:11.042922 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:11.042922 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:11.042922 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:11.042922 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:11.042922 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:11.042922 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:11.043833 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:11.043833 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:11.043833 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 14:20:11.043833 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 14:20:11.043833 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 14:20:11.043833 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 14:20:11.043833 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 14:20:11.043833 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 14:20:11.043833 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 07 14:20:11.043833 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:11.043833 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 242/495] 10.222.0.85 () {56 vars in 1093 bytes} [Tue Jun 7 14:20:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:11.045225 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71abba31-33f2-4ff8-873e-101d97f91f1a None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:11.057445 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73675b5e-1417-460e-9776-9be3c1180b85 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:11.084967 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:11.097291 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac4c249-1c92-474b-9c47-751fb5cf1e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:11.098098 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac4c249-1c92-474b-9c47-751fb5cf1e1c None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:11.098382 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac4c249-1c92-474b-9c47-751fb5cf1e1c None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:11.098697 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac4c249-1c92-474b-9c47-751fb5cf1e1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:11.109827 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac4c249-1c92-474b-9c47-751fb5cf1e1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:11.111309 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac4c249-1c92-474b-9c47-751fb5cf1e1c None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:11.121835 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:11.132390 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-dac4c249-1c92-474b-9c47-751fb5cf1e1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:11.132390 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:11.132390 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:11.132390 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:11.132390 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:11.132390 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:11.132390 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:11.132390 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:11.132390 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:11.132390 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:11.132390 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:11.132390 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:11.132390 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:20:11.132390 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:20:11.132390 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:20:11.139275 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:20:11.139275 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:11.139275 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:11.139275 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:11.139275 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:11.139275 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:11.139275 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:11.139275 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:11.139275 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:11.139275 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:11.139275 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:11.139275 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:11.139275 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:11.139275 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:11.139275 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:11.139275 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:11.139275 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:11.140113 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:20:11.140113 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:20:11.140113 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:11.140113 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:11.140113 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 254/496] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:20:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:11.140113 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45868e33-29f4-49dd-9295-29051e930563 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:11.140113 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45868e33-29f4-49dd-9295-29051e930563 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:11.140113 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45868e33-29f4-49dd-9295-29051e930563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:11.152846 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-45868e33-29f4-49dd-9295-29051e930563 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:20:11.154264 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-45868e33-29f4-49dd-9295-29051e930563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:20:11.165736 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71abba31-33f2-4ff8-873e-101d97f91f1a None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:11.201639 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0693ada3-15e1-45b3-8b9d-47c44f4633c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:11.202306 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0693ada3-15e1-45b3-8b9d-47c44f4633c5 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:11.202591 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0693ada3-15e1-45b3-8b9d-47c44f4633c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:11.203001 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0693ada3-15e1-45b3-8b9d-47c44f4633c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:11.203940 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0693ada3-15e1-45b3-8b9d-47c44f4633c5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:11.204449 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0693ada3-15e1-45b3-8b9d-47c44f4633c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:11.206080 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0693ada3-15e1-45b3-8b9d-47c44f4633c5 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:11.219729 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 243/497] 10.222.0.85 () {56 vars in 1103 bytes} [Tue Jun 7 14:20:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:11.219729 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0693ada3-15e1-45b3-8b9d-47c44f4633c5 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:11.246714 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 255/498] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:20:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:20:11.249603 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45868e33-29f4-49dd-9295-29051e930563 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:11.324284 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:11.327919 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0693ada3-15e1-45b3-8b9d-47c44f4633c5 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:11.362383 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d39af0-2525-40ca-b469-50846b15dc96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:11.362626 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45868e33-29f4-49dd-9295-29051e930563 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:11.363056 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d39af0-2525-40ca-b469-50846b15dc96 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:11.363346 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d39af0-2525-40ca-b469-50846b15dc96 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:11.363634 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d39af0-2525-40ca-b469-50846b15dc96 None admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:11.364522 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d39af0-2525-40ca-b469-50846b15dc96 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:11.364839 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d39af0-2525-40ca-b469-50846b15dc96 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:11.366282 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d39af0-2525-40ca-b469-50846b15dc96 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:11.392256 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:11.399253 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0973293b-8277-4705-86f8-2dc5d0665618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:11.399922 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0973293b-8277-4705-86f8-2dc5d0665618 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:11.400286 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0973293b-8277-4705-86f8-2dc5d0665618 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:11.400509 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0973293b-8277-4705-86f8-2dc5d0665618 None admin] PATH_INFO: `/v3/domains` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:11.401240 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0973293b-8277-4705-86f8-2dc5d0665618 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:11.401550 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0973293b-8277-4705-86f8-2dc5d0665618 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:11.402927 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0973293b-8277-4705-86f8-2dc5d0665618 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:11.406313 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 244/499] 10.222.0.85 () {56 vars in 1133 bytes} [Tue Jun 7 14:20:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:11.407157 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d39af0-2525-40ca-b469-50846b15dc96 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:11.415223 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 256/500] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jun 7 14:20:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:11.416054 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0973293b-8277-4705-86f8-2dc5d0665618 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:11.506330 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d39af0-2525-40ca-b469-50846b15dc96 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:11.512726 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0973293b-8277-4705-86f8-2dc5d0665618 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:11.541229 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbe7e53-5ee1-4da2-b66b-6a0f9a3f6912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:11.542021 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbe7e53-5ee1-4da2-b66b-6a0f9a3f6912 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:11.542306 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbe7e53-5ee1-4da2-b66b-6a0f9a3f6912 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:11.542589 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbe7e53-5ee1-4da2-b66b-6a0f9a3f6912 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:11.546539 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770c5a12-4f79-4ca7-9a53-cae37cf6438e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:11.547322 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770c5a12-4f79-4ca7-9a53-cae37cf6438e None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:11.547614 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770c5a12-4f79-4ca7-9a53-cae37cf6438e None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:11.547893 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770c5a12-4f79-4ca7-9a53-cae37cf6438e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:11.552576 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbe7e53-5ee1-4da2-b66b-6a0f9a3f6912 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:11.554053 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbe7e53-5ee1-4da2-b66b-6a0f9a3f6912 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:11.557959 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770c5a12-4f79-4ca7-9a53-cae37cf6438e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:20:11.565799 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-770c5a12-4f79-4ca7-9a53-cae37cf6438e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:20:11.565799 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:11.565799 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:11.565799 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:11.565799 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:11.565799 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:11.565799 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:11.565799 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:11.565799 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:11.565799 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:11.565799 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:11.565799 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:11.565799 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:20:11.565799 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:20:11.565799 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:20:11.565799 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:20:11.566664 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:20:11.566664 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:20:11.566664 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:20:11.566664 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:20:11.566664 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:11.566664 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:11.566664 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:11.566664 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:11.566664 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:11.566664 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:11.566664 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:11.566664 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:11.566664 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:11.566664 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:11.566664 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:11.566664 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:11.567424 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:11.567424 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:11.567424 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:11.567424 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:11.567424 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:20:11.567424 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:20:11.567424 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:20:11.567424 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:20:11.567424 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:20:11.567424 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:20:11.567424 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:20:11.567424 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:11.567424 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 257/501] 10.222.0.85 () {56 vars in 1102 bytes} [Tue Jun 7 14:20:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:11.568680 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aadb30-7009-4fc0-920d-c96e31821998 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:11.568968 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aadb30-7009-4fc0-920d-c96e31821998 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:11.569249 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aadb30-7009-4fc0-920d-c96e31821998 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:11.570550 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 258/502] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:20:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:20:11.572498 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aadb30-7009-4fc0-920d-c96e31821998 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:11.572595 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-1cbe7e53-5ee1-4da2-b66b-6a0f9a3f6912 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:11.572595 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:11.572595 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:11.572595 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:11.572595 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:11.572595 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:11.572595 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:11.572595 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:11.572595 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:11.572595 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:11.572595 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:11.572595 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:11.572595 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:20:11.572595 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:20:11.572595 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:20:11.573556 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:20:11.573556 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:11.573556 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:11.573556 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:11.573556 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:11.573556 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:11.573556 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:11.573556 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:11.573556 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:11.573556 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:11.573556 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:11.573556 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:11.573556 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:11.573556 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:11.573556 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:11.573556 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:11.573556 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:11.574916 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:20:11.574916 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:20:11.574916 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:11.574916 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:11.574916 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 245/503] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:20:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:11.574916 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbe7e53-5ee1-4da2-b66b-6a0f9a3f6912 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:11.586788 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:11.626824 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:11.676929 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbe7e53-5ee1-4da2-b66b-6a0f9a3f6912 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:11.678219 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aadb30-7009-4fc0-920d-c96e31821998 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:11.711377 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb5bf88-a7da-4cbc-b5b1-7539504b1789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:11.712056 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17047a7a-7dd3-4a63-a438-71cc47a6b631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:11.712205 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb5bf88-a7da-4cbc-b5b1-7539504b1789 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:11.712322 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17047a7a-7dd3-4a63-a438-71cc47a6b631 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:11.712466 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb5bf88-a7da-4cbc-b5b1-7539504b1789 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:11.712631 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17047a7a-7dd3-4a63-a438-71cc47a6b631 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:11.712745 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb5bf88-a7da-4cbc-b5b1-7539504b1789 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:11.712904 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17047a7a-7dd3-4a63-a438-71cc47a6b631 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:11.722686 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb5bf88-a7da-4cbc-b5b1-7539504b1789 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:11.724540 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb5bf88-a7da-4cbc-b5b1-7539504b1789 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:11.740171 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-dbb5bf88-a7da-4cbc-b5b1-7539504b1789 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 07 14:20:11.740171 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:11.740171 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:11.740171 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:11.740171 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:11.740171 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:11.740171 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:11.740171 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:11.740171 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:11.740171 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:11.740171 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:11.740171 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:11.740171 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 14:20:11.740171 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 14:20:11.740171 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 14:20:11.740171 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 14:20:11.741238 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:11.741238 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:11.741238 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 14:20:11.741238 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:20:11.741238 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 14:20:11.741238 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:20:11.741238 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:11.741238 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:11.741238 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:11.741238 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:11.741238 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:11.741238 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:11.741238 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:11.741238 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:11.741238 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:11.741238 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:11.741238 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:11.741238 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:11.742259 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:11.742259 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:11.742259 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 14:20:11.742259 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 14:20:11.742259 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 14:20:11.742259 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 14:20:11.742259 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 14:20:11.742259 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 14:20:11.742259 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 07 14:20:11.742259 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:11.742259 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 246/504] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jun 7 14:20:11 2022] GET /identity/v3/users/cinder => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:11.743149 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb5bf88-a7da-4cbc-b5b1-7539504b1789 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:11.784915 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17047a7a-7dd3-4a63-a438-71cc47a6b631 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:11.786295 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17047a7a-7dd3-4a63-a438-71cc47a6b631 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:11.829083 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:11.840217 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:11.862736 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb5bf88-a7da-4cbc-b5b1-7539504b1789 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:11.891749 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 259/505] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jun 7 14:20:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 14:20:11.893097 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963e6e82-c7f6-4f5a-be77-547d92f9b315 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:11.893395 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963e6e82-c7f6-4f5a-be77-547d92f9b315 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:11.893683 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963e6e82-c7f6-4f5a-be77-547d92f9b315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:11.901709 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de517fae-b43d-4c1c-8ea7-e312cbcde07d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:11.902381 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de517fae-b43d-4c1c-8ea7-e312cbcde07d None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:11.902668 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de517fae-b43d-4c1c-8ea7-e312cbcde07d None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:11.902948 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de517fae-b43d-4c1c-8ea7-e312cbcde07d None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:11.903705 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de517fae-b43d-4c1c-8ea7-e312cbcde07d None admin] RBAC: Adding query filter params (name=service) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:11.904017 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de517fae-b43d-4c1c-8ea7-e312cbcde07d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:11.905449 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de517fae-b43d-4c1c-8ea7-e312cbcde07d None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:11.911584 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-963e6e82-c7f6-4f5a-be77-547d92f9b315 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:20:11.912803 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-963e6e82-c7f6-4f5a-be77-547d92f9b315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:20:11.913737 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 247/506] 10.222.0.85 () {56 vars in 1103 bytes} [Tue Jun 7 14:20:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:11.914581 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de517fae-b43d-4c1c-8ea7-e312cbcde07d None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:11.982562 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 260/507] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:20:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:20:11.983411 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963e6e82-c7f6-4f5a-be77-547d92f9b315 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:12.025038 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de517fae-b43d-4c1c-8ea7-e312cbcde07d None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:12.057037 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:12.062594 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb089e5-cc45-4b00-a944-893297fecd38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:12.063256 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb089e5-cc45-4b00-a944-893297fecd38 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:12.063537 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb089e5-cc45-4b00-a944-893297fecd38 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:12.063812 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb089e5-cc45-4b00-a944-893297fecd38 None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:12.064573 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb089e5-cc45-4b00-a944-893297fecd38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:12.064880 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb089e5-cc45-4b00-a944-893297fecd38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:12.066268 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb089e5-cc45-4b00-a944-893297fecd38 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:12.067297 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:12.078808 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 248/508] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jun 7 14:20:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:12.079595 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb089e5-cc45-4b00-a944-893297fecd38 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:12.087207 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963e6e82-c7f6-4f5a-be77-547d92f9b315 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:12.121061 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3c75d7-1e13-4e30-9a90-f1c2809bffef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:12.121741 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3c75d7-1e13-4e30-9a90-f1c2809bffef None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:12.122041 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3c75d7-1e13-4e30-9a90-f1c2809bffef None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:12.122571 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3c75d7-1e13-4e30-9a90-f1c2809bffef None admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:12.123572 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3c75d7-1e13-4e30-9a90-f1c2809bffef None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:12.123948 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3c75d7-1e13-4e30-9a90-f1c2809bffef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:12.125874 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3c75d7-1e13-4e30-9a90-f1c2809bffef None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:12.162167 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 261/509] 10.222.0.85 () {56 vars in 1137 bytes} [Tue Jun 7 14:20:11 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:12.163010 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3c75d7-1e13-4e30-9a90-f1c2809bffef None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:12.183346 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb089e5-cc45-4b00-a944-893297fecd38 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:12.220032 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a8832d-1f63-4b7c-9fc0-39cc081572bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:12.220867 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a8832d-1f63-4b7c-9fc0-39cc081572bb None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:12.221154 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a8832d-1f63-4b7c-9fc0-39cc081572bb None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:12.221430 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a8832d-1f63-4b7c-9fc0-39cc081572bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:12.231700 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a8832d-1f63-4b7c-9fc0-39cc081572bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:12.233201 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a8832d-1f63-4b7c-9fc0-39cc081572bb None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:12.251450 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-92a8832d-1f63-4b7c-9fc0-39cc081572bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:12.251450 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:12.251450 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:12.251450 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:12.251450 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:12.251450 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:12.251450 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:12.251450 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:12.251450 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:12.251450 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:12.251450 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:12.251450 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:12.251450 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:20:12.251450 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:20:12.251450 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:20:12.252450 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:20:12.252450 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:12.252450 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:12.252450 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:12.252450 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:12.252450 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:12.252450 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:12.252450 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:12.252450 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:12.252450 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:12.252450 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:12.252450 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:12.252450 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:12.252450 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:12.252450 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:12.252450 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:12.252450 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:12.253297 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:20:12.253297 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:20:12.253297 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:12.253297 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:12.253297 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 249/510] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:20:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:12.253578 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a8832d-1f63-4b7c-9fc0-39cc081572bb None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:12.270056 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3c75d7-1e13-4e30-9a90-f1c2809bffef None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:12.298381 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:12.305520 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5609b881-ea77-4bf1-a146-5e77ff7ae240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:12.309735 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5609b881-ea77-4bf1-a146-5e77ff7ae240 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:12.309735 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5609b881-ea77-4bf1-a146-5e77ff7ae240 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:12.309735 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5609b881-ea77-4bf1-a146-5e77ff7ae240 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:12.314045 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:12.318115 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5609b881-ea77-4bf1-a146-5e77ff7ae240 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:12.319584 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5609b881-ea77-4bf1-a146-5e77ff7ae240 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:12.339048 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-5609b881-ea77-4bf1-a146-5e77ff7ae240 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:12.339048 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:12.339048 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:12.339048 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:12.339048 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:12.339048 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:12.339048 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:12.339048 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:12.339048 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:12.339048 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:12.339048 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:12.339048 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:12.339048 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:20:12.339048 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:20:12.339048 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:20:12.340078 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:20:12.340078 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:12.340078 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:12.340078 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:12.340078 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:12.340078 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:12.340078 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:12.340078 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:12.340078 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:12.340078 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:12.340078 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:12.340078 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:12.340078 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:12.340078 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:12.340078 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:12.340078 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:12.340078 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:12.344886 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:20:12.344886 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:20:12.344886 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:12.344886 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:12.344886 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 262/511] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:20:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:12.344886 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5609b881-ea77-4bf1-a146-5e77ff7ae240 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:12.357386 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a8832d-1f63-4b7c-9fc0-39cc081572bb None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:12.391762 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b8c620-cfd7-44b1-9089-2c603594cce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:12.392587 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b8c620-cfd7-44b1-9089-2c603594cce8 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:12.392873 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b8c620-cfd7-44b1-9089-2c603594cce8 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:12.393151 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b8c620-cfd7-44b1-9089-2c603594cce8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:12.405476 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b8c620-cfd7-44b1-9089-2c603594cce8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:12.407012 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b8c620-cfd7-44b1-9089-2c603594cce8 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:12.425186 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-35b8c620-cfd7-44b1-9089-2c603594cce8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:12.425186 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:12.425186 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:12.425186 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:12.425186 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:12.425186 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:12.425186 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:12.425186 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:12.425186 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:12.425186 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:12.425186 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:12.425186 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:12.425186 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:20:12.425186 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:20:12.425186 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:20:12.426082 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:20:12.426082 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:12.426082 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:12.426082 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:12.426082 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:12.426082 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:12.426082 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:12.426082 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:12.426082 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:12.426082 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:12.426082 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:12.426082 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:12.426082 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:12.426082 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:12.426082 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:12.426082 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:12.426082 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:12.426923 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:20:12.426923 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:20:12.426923 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:12.426923 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:12.426923 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 250/512] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:20:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:12.427473 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b8c620-cfd7-44b1-9089-2c603594cce8 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:12.448815 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5609b881-ea77-4bf1-a146-5e77ff7ae240 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:12.487215 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575c7895-4cae-48f4-825f-e77782bf212d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:12.488016 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575c7895-4cae-48f4-825f-e77782bf212d None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:12.488309 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575c7895-4cae-48f4-825f-e77782bf212d None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:12.488624 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575c7895-4cae-48f4-825f-e77782bf212d None admin] PATH_INFO: `/v3/users/glance` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:12.497440 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575c7895-4cae-48f4-825f-e77782bf212d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:12.498889 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575c7895-4cae-48f4-825f-e77782bf212d None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:12.514820 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-575c7895-4cae-48f4-825f-e77782bf212d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 07 14:20:12.514820 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:12.514820 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:12.514820 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:12.514820 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:12.514820 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:12.514820 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:12.514820 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:12.514820 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:12.514820 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:12.514820 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:12.514820 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:12.514820 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 14:20:12.514820 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 14:20:12.514820 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 14:20:12.514820 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 14:20:12.515878 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:12.515878 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:12.515878 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 14:20:12.515878 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:20:12.515878 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 14:20:12.515878 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:20:12.515878 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:12.515878 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:12.515878 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:12.515878 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:12.515878 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:12.515878 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:12.515878 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:12.515878 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:12.515878 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:12.515878 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:12.515878 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:12.515878 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:12.516858 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:12.516858 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:12.516858 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 14:20:12.516858 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 14:20:12.516858 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 14:20:12.516858 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 14:20:12.516858 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 14:20:12.516858 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 14:20:12.516858 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 07 14:20:12.516858 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:12.516858 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 263/513] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jun 7 14:20:12 2022] GET /identity/v3/users/glance => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:12.516858 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575c7895-4cae-48f4-825f-e77782bf212d None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:12.532689 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:12.539570 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b8c620-cfd7-44b1-9089-2c603594cce8 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:12.566445 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:12.574125 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2aebd4-84e4-4cfc-a96f-fef83a80539b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:12.574779 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2aebd4-84e4-4cfc-a96f-fef83a80539b None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:12.575065 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2aebd4-84e4-4cfc-a96f-fef83a80539b None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:12.575345 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2aebd4-84e4-4cfc-a96f-fef83a80539b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:12.620548 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575c7895-4cae-48f4-825f-e77782bf212d None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:12.645501 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2aebd4-84e4-4cfc-a96f-fef83a80539b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:12.646980 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2aebd4-84e4-4cfc-a96f-fef83a80539b None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:12.655363 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953b80a9-0486-4816-9de7-0be4dfa99d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:12.656035 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953b80a9-0486-4816-9de7-0be4dfa99d7e None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:12.656501 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953b80a9-0486-4816-9de7-0be4dfa99d7e None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:12.656988 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953b80a9-0486-4816-9de7-0be4dfa99d7e None admin] PATH_INFO: `/v3/domains` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:12.657764 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953b80a9-0486-4816-9de7-0be4dfa99d7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:12.658076 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953b80a9-0486-4816-9de7-0be4dfa99d7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:12.659455 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953b80a9-0486-4816-9de7-0be4dfa99d7e None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:12.671290 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 264/514] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jun 7 14:20:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:12.672125 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953b80a9-0486-4816-9de7-0be4dfa99d7e None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:12.743134 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:12.750445 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 251/515] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jun 7 14:20:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 14:20:12.751338 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2aebd4-84e4-4cfc-a96f-fef83a80539b None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:12.777566 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953b80a9-0486-4816-9de7-0be4dfa99d7e None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:12.788948 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:12.811832 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628706a7-8575-41bc-9ea2-ed8b75683953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:12.813192 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628706a7-8575-41bc-9ea2-ed8b75683953 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:12.813192 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628706a7-8575-41bc-9ea2-ed8b75683953 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:12.813192 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628706a7-8575-41bc-9ea2-ed8b75683953 None admin] PATH_INFO: `/v3/domains` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:12.813864 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628706a7-8575-41bc-9ea2-ed8b75683953 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:12.814171 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628706a7-8575-41bc-9ea2-ed8b75683953 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:12.815541 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628706a7-8575-41bc-9ea2-ed8b75683953 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:12.827535 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 265/516] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jun 7 14:20:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:12.828302 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628706a7-8575-41bc-9ea2-ed8b75683953 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:12.857850 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2aebd4-84e4-4cfc-a96f-fef83a80539b None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:12.893210 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf29dc0-7d2b-47b4-a8d9-69733b03f4c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:12.893887 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf29dc0-7d2b-47b4-a8d9-69733b03f4c9 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:12.894174 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf29dc0-7d2b-47b4-a8d9-69733b03f4c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:12.894449 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf29dc0-7d2b-47b4-a8d9-69733b03f4c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:12.895218 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf29dc0-7d2b-47b4-a8d9-69733b03f4c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:12.895524 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf29dc0-7d2b-47b4-a8d9-69733b03f4c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:12.896957 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf29dc0-7d2b-47b4-a8d9-69733b03f4c9 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:12.911635 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 252/517] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jun 7 14:20:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:12.912529 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf29dc0-7d2b-47b4-a8d9-69733b03f4c9 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:12.935654 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628706a7-8575-41bc-9ea2-ed8b75683953 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:12.977624 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62664c70-5595-4dea-8639-5d9ec21806f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:12.978321 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62664c70-5595-4dea-8639-5d9ec21806f0 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:12.978607 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62664c70-5595-4dea-8639-5d9ec21806f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:12.978884 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62664c70-5595-4dea-8639-5d9ec21806f0 None admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:12.979751 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62664c70-5595-4dea-8639-5d9ec21806f0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:12.980063 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62664c70-5595-4dea-8639-5d9ec21806f0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:12.981595 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62664c70-5595-4dea-8639-5d9ec21806f0 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:13.001659 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 266/518] 10.222.0.85 () {56 vars in 1137 bytes} [Tue Jun 7 14:20:12 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:13.002553 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62664c70-5595-4dea-8639-5d9ec21806f0 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:13.034628 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:13.100052 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:13.120255 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62664c70-5595-4dea-8639-5d9ec21806f0 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:13.143910 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf29dc0-7d2b-47b4-a8d9-69733b03f4c9 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:13.154827 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07fc285-d30f-4d09-9ae3-925c6dd79bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:13.155615 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07fc285-d30f-4d09-9ae3-925c6dd79bed None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:13.155895 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07fc285-d30f-4d09-9ae3-925c6dd79bed None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:13.156174 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07fc285-d30f-4d09-9ae3-925c6dd79bed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:13.166773 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07fc285-d30f-4d09-9ae3-925c6dd79bed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:13.168221 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07fc285-d30f-4d09-9ae3-925c6dd79bed None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:13.180742 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee354b1-bc73-49dc-a9aa-dd223f3bb479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:13.181534 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee354b1-bc73-49dc-a9aa-dd223f3bb479 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:13.181823 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee354b1-bc73-49dc-a9aa-dd223f3bb479 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:13.182101 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee354b1-bc73-49dc-a9aa-dd223f3bb479 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:13.185473 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-c07fc285-d30f-4d09-9ae3-925c6dd79bed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:13.185473 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:13.185473 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:13.185473 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:13.185473 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:13.185473 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:13.185473 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:13.185473 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:13.185473 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:13.185473 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:13.185473 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:13.185473 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:13.185473 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:20:13.185473 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:20:13.185473 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:20:13.186453 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:20:13.186453 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:13.186453 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:13.186453 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:13.186453 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:13.186453 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:13.186453 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:13.186453 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:13.186453 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:13.186453 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:13.186453 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:13.186453 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:13.186453 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:13.186453 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:13.186453 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:13.186453 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:13.186453 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:13.187470 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:20:13.187470 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:20:13.187470 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:13.187470 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:13.187470 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 267/519] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:20:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:13.187953 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07fc285-d30f-4d09-9ae3-925c6dd79bed None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:13.194705 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee354b1-bc73-49dc-a9aa-dd223f3bb479 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:13.196189 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee354b1-bc73-49dc-a9aa-dd223f3bb479 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:13.211704 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-dee354b1-bc73-49dc-a9aa-dd223f3bb479 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:20:13.211704 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:13.211704 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:13.211704 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:13.211704 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:13.211704 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:13.211704 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:13.211704 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:13.211704 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:13.211704 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:13.211704 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:13.211704 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:13.211704 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 14:20:13.211704 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 14:20:13.211704 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 14:20:13.212665 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 14:20:13.212665 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:13.212665 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:13.212665 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:13.212665 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:13.212665 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:13.212665 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:13.212665 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:13.212665 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:13.212665 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:13.212665 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:13.212665 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:13.212665 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:13.212665 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:13.212665 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:13.212665 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:13.212665 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:13.213592 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 14:20:13.213592 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 14:20:13.213592 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 14:20:13.213592 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 14:20:13.213592 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 14:20:13.213592 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 14:20:13.213592 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:20:13.213592 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:13.213592 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 253/520] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jun 7 14:20:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:13.214071 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee354b1-bc73-49dc-a9aa-dd223f3bb479 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:13.272123 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:13.291030 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07fc285-d30f-4d09-9ae3-925c6dd79bed None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:13.314753 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee354b1-bc73-49dc-a9aa-dd223f3bb479 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:13.324867 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f69345-13e3-42a9-9a2f-9ede211b3d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:13.325670 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f69345-13e3-42a9-9a2f-9ede211b3d51 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:13.325948 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f69345-13e3-42a9-9a2f-9ede211b3d51 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:13.326225 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f69345-13e3-42a9-9a2f-9ede211b3d51 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:13.329127 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:13.333615 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f69345-13e3-42a9-9a2f-9ede211b3d51 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:13.335070 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f69345-13e3-42a9-9a2f-9ede211b3d51 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:13.348918 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b78e7e8-c533-443f-a419-439cd9a6e13f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:13.349582 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b78e7e8-c533-443f-a419-439cd9a6e13f None admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:13.349783 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-03f69345-13e3-42a9-9a2f-9ede211b3d51 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 07 14:20:13.349783 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:13.349783 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:13.349783 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:13.349783 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:13.349783 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:13.349783 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:13.349783 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:13.349783 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:13.349783 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:13.349783 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:13.349783 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:13.349783 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 14:20:13.349783 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 14:20:13.349783 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 14:20:13.349783 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 14:20:13.351151 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:13.351151 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:13.351151 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 14:20:13.351151 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:20:13.351151 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 14:20:13.351151 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:20:13.351151 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:13.351151 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:13.351151 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:13.351151 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:13.351151 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:13.351151 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:13.351151 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:13.351151 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:13.351151 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:13.351151 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:13.351151 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:13.351151 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:13.352073 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:13.352073 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:13.352073 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 14:20:13.352073 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 14:20:13.352073 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 14:20:13.352073 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 14:20:13.352073 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 14:20:13.352073 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 14:20:13.352073 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 07 14:20:13.352073 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:13.352633 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b78e7e8-c533-443f-a419-439cd9a6e13f None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:13.352633 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b78e7e8-c533-443f-a419-439cd9a6e13f None admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:13.352633 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b78e7e8-c533-443f-a419-439cd9a6e13f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:13.352822 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 268/521] 10.222.0.85 () {56 vars in 1102 bytes} [Tue Jun 7 14:20:13 2022] GET /identity/v3/users/neutron => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:13.352822 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f69345-13e3-42a9-9a2f-9ede211b3d51 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:13.352954 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b78e7e8-c533-443f-a419-439cd9a6e13f None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:13.454810 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f69345-13e3-42a9-9a2f-9ede211b3d51 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:13.460074 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 254/522] 10.222.0.85 () {60 vars in 1132 bytes} [Tue Jun 7 14:20:13 2022] POST /identity/v3/users => generated 247 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:20:13.460942 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b78e7e8-c533-443f-a419-439cd9a6e13f None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:13.504230 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaaa6d7-195e-4435-9b56-ea48b4e29410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:13.504937 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaaa6d7-195e-4435-9b56-ea48b4e29410 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:13.505232 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaaa6d7-195e-4435-9b56-ea48b4e29410 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:13.505514 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaaa6d7-195e-4435-9b56-ea48b4e29410 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:13.517574 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:13.587645 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b78e7e8-c533-443f-a419-439cd9a6e13f None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:13.590020 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaaa6d7-195e-4435-9b56-ea48b4e29410 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:13.591522 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaaa6d7-195e-4435-9b56-ea48b4e29410 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:13.628397 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dcf9cb-ad87-4bb2-9a4a-195780d6e093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:13.629427 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dcf9cb-ad87-4bb2-9a4a-195780d6e093 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:13.629714 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dcf9cb-ad87-4bb2-9a4a-195780d6e093 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:13.629992 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dcf9cb-ad87-4bb2-9a4a-195780d6e093 None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:13.630742 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dcf9cb-ad87-4bb2-9a4a-195780d6e093 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:13.631055 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dcf9cb-ad87-4bb2-9a4a-195780d6e093 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:13.632538 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dcf9cb-ad87-4bb2-9a4a-195780d6e093 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:13.651737 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 255/523] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jun 7 14:20:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:13.651737 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dcf9cb-ad87-4bb2-9a4a-195780d6e093 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:13.688420 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:13.708189 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 269/524] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jun 7 14:20:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 14:20:13.708637 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaaa6d7-195e-4435-9b56-ea48b4e29410 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:13.755647 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:13.771198 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dcf9cb-ad87-4bb2-9a4a-195780d6e093 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:13.811033 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3299d27-bc25-4e75-bc1a-86ed6652d7b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:13.811705 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3299d27-bc25-4e75-bc1a-86ed6652d7b3 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:13.812102 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3299d27-bc25-4e75-bc1a-86ed6652d7b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:13.812410 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3299d27-bc25-4e75-bc1a-86ed6652d7b3 None admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:13.813350 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3299d27-bc25-4e75-bc1a-86ed6652d7b3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:13.813668 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3299d27-bc25-4e75-bc1a-86ed6652d7b3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:13.815105 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3299d27-bc25-4e75-bc1a-86ed6652d7b3 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:13.837892 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaaa6d7-195e-4435-9b56-ea48b4e29410 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:13.849880 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 256/525] 10.222.0.85 () {56 vars in 1148 bytes} [Tue Jun 7 14:20:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:13.851640 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3299d27-bc25-4e75-bc1a-86ed6652d7b3 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:13.882100 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0d81b4-7be9-47b5-8194-8e9f8de34464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:13.883254 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0d81b4-7be9-47b5-8194-8e9f8de34464 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:13.883540 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0d81b4-7be9-47b5-8194-8e9f8de34464 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:13.883823 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0d81b4-7be9-47b5-8194-8e9f8de34464 None admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:13.884699 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0d81b4-7be9-47b5-8194-8e9f8de34464 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:13.885018 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0d81b4-7be9-47b5-8194-8e9f8de34464 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:13.886459 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0d81b4-7be9-47b5-8194-8e9f8de34464 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:13.907133 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 270/526] 10.222.0.85 () {56 vars in 1139 bytes} [Tue Jun 7 14:20:13 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:13.907987 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0d81b4-7be9-47b5-8194-8e9f8de34464 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:13.967033 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:13.977568 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3299d27-bc25-4e75-bc1a-86ed6652d7b3 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:14.013594 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a039ab-29d4-4414-90fa-22469c81bb8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:14.013826 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:14.014396 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a039ab-29d4-4414-90fa-22469c81bb8a None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:14.014719 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a039ab-29d4-4414-90fa-22469c81bb8a None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:14.014958 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a039ab-29d4-4414-90fa-22469c81bb8a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:14.018059 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0d81b4-7be9-47b5-8194-8e9f8de34464 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:14.021987 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a039ab-29d4-4414-90fa-22469c81bb8a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:14.023463 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a039ab-29d4-4414-90fa-22469c81bb8a None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:14.039099 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-81a039ab-29d4-4414-90fa-22469c81bb8a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:20:14.039099 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:14.039099 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:14.039099 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:14.039099 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:14.039099 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:14.039099 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:14.039099 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:14.039099 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:14.039099 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:14.039099 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:14.039099 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:14.039099 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 14:20:14.039099 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 14:20:14.039099 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 14:20:14.040069 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 14:20:14.040069 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:14.040069 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:14.040069 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:14.040069 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:14.040069 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:14.040069 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:14.040069 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:14.040069 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:14.040069 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:14.040069 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:14.040069 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:14.040069 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:14.040069 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:14.040069 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:14.040069 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:14.040069 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:14.040947 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 14:20:14.040947 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 14:20:14.040947 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 14:20:14.040947 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 14:20:14.040947 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 14:20:14.040947 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 14:20:14.040947 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:20:14.040947 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:14.040947 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 257/527] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jun 7 14:20:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:14.041880 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a039ab-29d4-4414-90fa-22469c81bb8a None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:14.061274 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093a1b8b-c89a-4da8-94a4-fd1fcc18231b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:14.062065 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093a1b8b-c89a-4da8-94a4-fd1fcc18231b None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:14.062349 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093a1b8b-c89a-4da8-94a4-fd1fcc18231b None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:14.062624 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093a1b8b-c89a-4da8-94a4-fd1fcc18231b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:14.073449 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093a1b8b-c89a-4da8-94a4-fd1fcc18231b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:14.074898 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093a1b8b-c89a-4da8-94a4-fd1fcc18231b None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:14.092960 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-093a1b8b-c89a-4da8-94a4-fd1fcc18231b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:14.092960 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:14.092960 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:14.092960 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:14.092960 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:14.092960 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:14.092960 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:14.092960 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:14.092960 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:14.092960 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:14.092960 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:14.092960 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:14.092960 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:20:14.092960 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:20:14.092960 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:20:14.094057 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:20:14.094057 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:14.094057 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:14.094057 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:14.094057 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:14.094057 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:14.094057 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:14.094057 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:14.094057 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:14.094057 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:14.094057 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:14.094057 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:14.094057 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:14.094057 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:14.094057 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:14.094057 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:14.094057 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:14.095082 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:20:14.095082 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:20:14.095082 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:14.095082 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:14.095082 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 271/528] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:20:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:14.095412 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093a1b8b-c89a-4da8-94a4-fd1fcc18231b None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:14.165949 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a039ab-29d4-4414-90fa-22469c81bb8a None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:14.201636 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbec92c6-daf8-452f-9afe-7a5e42658f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:14.202373 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbec92c6-daf8-452f-9afe-7a5e42658f57 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:14.202772 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbec92c6-daf8-452f-9afe-7a5e42658f57 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:14.203157 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbec92c6-daf8-452f-9afe-7a5e42658f57 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:14.204139 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbec92c6-daf8-452f-9afe-7a5e42658f57 None admin] RBAC: Adding query filter params (role.id=17516844e8b947b389d62f50718b852d, scope.project.id=724785eb74e44daf8736fd01fc0d1839, user.id=8e857db4902e42989d0d040bbc8517c8) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:14.204597 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbec92c6-daf8-452f-9afe-7a5e42658f57 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:14.206140 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbec92c6-daf8-452f-9afe-7a5e42658f57 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:14.212762 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 258/529] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jun 7 14:20:14 2022] GET /identity/v3/role_assignments?user.id=8e857db4902e42989d0d040bbc8517c8&scope.project.id=724785eb74e44daf8736fd01fc0d1839&role.id=17516844e8b947b389d62f50718b852d => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:14.213525 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbec92c6-daf8-452f-9afe-7a5e42658f57 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:14.232812 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093a1b8b-c89a-4da8-94a4-fd1fcc18231b None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:14.269085 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:14.300146 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ec338a-1330-4047-bd93-48c2c8113004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:14.301391 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ec338a-1330-4047-bd93-48c2c8113004 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:14.302097 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ec338a-1330-4047-bd93-48c2c8113004 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:14.302508 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ec338a-1330-4047-bd93-48c2c8113004 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:14.332162 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:14.376709 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbec92c6-daf8-452f-9afe-7a5e42658f57 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:14.396465 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ec338a-1330-4047-bd93-48c2c8113004 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:14.398441 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ec338a-1330-4047-bd93-48c2c8113004 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:14.424077 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60c5428-aed8-4d94-881e-f23db092c3d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:14.424999 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60c5428-aed8-4d94-881e-f23db092c3d9 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:14.425417 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60c5428-aed8-4d94-881e-f23db092c3d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:14.425799 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60c5428-aed8-4d94-881e-f23db092c3d9 None admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:14.426708 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60c5428-aed8-4d94-881e-f23db092c3d9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:14.427503 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60c5428-aed8-4d94-881e-f23db092c3d9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:14.430112 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60c5428-aed8-4d94-881e-f23db092c3d9 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:14.449277 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 259/530] 10.222.0.85 () {56 vars in 1148 bytes} [Tue Jun 7 14:20:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:14.450523 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60c5428-aed8-4d94-881e-f23db092c3d9 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:14.518855 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 272/531] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jun 7 14:20:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 14:20:14.520058 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ec338a-1330-4047-bd93-48c2c8113004 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:14.575359 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:14.587112 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60c5428-aed8-4d94-881e-f23db092c3d9 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:14.620902 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:14.631409 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ec338a-1330-4047-bd93-48c2c8113004 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:14.637320 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a08d24-0418-4cd5-9ab3-dc56050b035f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:14.637320 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a08d24-0418-4cd5-9ab3-dc56050b035f None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:14.637320 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a08d24-0418-4cd5-9ab3-dc56050b035f None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:14.637320 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a08d24-0418-4cd5-9ab3-dc56050b035f None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:14.637671 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a08d24-0418-4cd5-9ab3-dc56050b035f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:14.637671 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a08d24-0418-4cd5-9ab3-dc56050b035f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:14.639171 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a08d24-0418-4cd5-9ab3-dc56050b035f None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:14.660913 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 260/532] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jun 7 14:20:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:14.662243 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a08d24-0418-4cd5-9ab3-dc56050b035f None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:14.676729 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3826d2a2-98a6-4ba2-9be7-cf1d6f79e5f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:14.677831 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3826d2a2-98a6-4ba2-9be7-cf1d6f79e5f2 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:14.678247 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3826d2a2-98a6-4ba2-9be7-cf1d6f79e5f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:14.678721 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3826d2a2-98a6-4ba2-9be7-cf1d6f79e5f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:14.680412 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3826d2a2-98a6-4ba2-9be7-cf1d6f79e5f2 None admin] RBAC: Adding query filter params (role.id=17516844e8b947b389d62f50718b852d, scope.project.id=724785eb74e44daf8736fd01fc0d1839, user.id=89d81f597cb44c56a966537089bcce4e) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:14.680888 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3826d2a2-98a6-4ba2-9be7-cf1d6f79e5f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:14.682539 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3826d2a2-98a6-4ba2-9be7-cf1d6f79e5f2 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:14.696612 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 273/533] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jun 7 14:20:14 2022] GET /identity/v3/role_assignments?user.id=89d81f597cb44c56a966537089bcce4e&scope.project.id=724785eb74e44daf8736fd01fc0d1839&role.id=17516844e8b947b389d62f50718b852d => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:14.698015 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3826d2a2-98a6-4ba2-9be7-cf1d6f79e5f2 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:14.853597 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a08d24-0418-4cd5-9ab3-dc56050b035f None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:14.871884 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3826d2a2-98a6-4ba2-9be7-cf1d6f79e5f2 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:14.884777 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:14.908664 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a268de2c-b6b7-4b86-a21f-72437c1046cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:14.910206 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a268de2c-b6b7-4b86-a21f-72437c1046cb None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:14.910752 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a268de2c-b6b7-4b86-a21f-72437c1046cb None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:14.911236 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a268de2c-b6b7-4b86-a21f-72437c1046cb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:14.920274 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a268de2c-b6b7-4b86-a21f-72437c1046cb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:14.922767 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a268de2c-b6b7-4b86-a21f-72437c1046cb None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:14.947573 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-a268de2c-b6b7-4b86-a21f-72437c1046cb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:20:14.947573 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:14.947573 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:14.947573 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:14.947573 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:14.947573 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:14.947573 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:14.947573 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:14.947573 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:14.947573 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:14.947573 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:14.947573 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:14.947573 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 14:20:14.947573 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 14:20:14.947573 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 14:20:14.948459 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 14:20:14.948459 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:14.948459 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:14.948459 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:14.948459 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:14.948459 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:14.948459 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:14.948459 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:14.948459 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:14.948459 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:14.948459 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:14.948459 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:14.948459 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:14.948459 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:14.948459 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:14.948459 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:14.948459 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:14.949366 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 14:20:14.949366 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 14:20:14.949366 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 14:20:14.949366 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 14:20:14.949366 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 14:20:14.949366 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 14:20:14.949366 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:20:14.949366 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:14.953156 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 274/534] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jun 7 14:20:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:14.953837 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151d1dcc-4d03-47f1-b9b3-ff0e6dfeac55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:14.954761 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151d1dcc-4d03-47f1-b9b3-ff0e6dfeac55 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:14.955051 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151d1dcc-4d03-47f1-b9b3-ff0e6dfeac55 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:14.955403 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151d1dcc-4d03-47f1-b9b3-ff0e6dfeac55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:14.959456 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:14.965725 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a268de2c-b6b7-4b86-a21f-72437c1046cb None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:14.976637 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151d1dcc-4d03-47f1-b9b3-ff0e6dfeac55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:14.978257 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151d1dcc-4d03-47f1-b9b3-ff0e6dfeac55 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:14.998667 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-151d1dcc-4d03-47f1-b9b3-ff0e6dfeac55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:14.998667 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:14.998667 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:14.998667 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:14.998667 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:14.998667 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:14.998667 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:14.998667 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:14.998667 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:14.998667 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:14.998667 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:14.998667 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:14.998667 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:20:14.998667 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:20:14.998667 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:20:14.999592 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:20:14.999592 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:14.999592 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:14.999592 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:14.999592 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:14.999592 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:14.999592 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:14.999592 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:14.999592 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:14.999592 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:14.999592 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:14.999592 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:14.999592 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:14.999592 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:14.999592 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:14.999592 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:14.999592 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:15.000461 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:20:15.000461 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:20:15.000461 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:15.000461 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:15.000461 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 261/535] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:20:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:15.006433 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151d1dcc-4d03-47f1-b9b3-ff0e6dfeac55 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:15.096352 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a268de2c-b6b7-4b86-a21f-72437c1046cb None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:15.162677 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4453308-5bbf-47a6-a5ec-4b4b7f393a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:15.164984 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4453308-5bbf-47a6-a5ec-4b4b7f393a4c None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:15.166109 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4453308-5bbf-47a6-a5ec-4b4b7f393a4c None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:15.166630 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4453308-5bbf-47a6-a5ec-4b4b7f393a4c None admin] PATH_INFO: `/v3/projects` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:15.167787 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4453308-5bbf-47a6-a5ec-4b4b7f393a4c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:15.169439 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4453308-5bbf-47a6-a5ec-4b4b7f393a4c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:15.171149 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4453308-5bbf-47a6-a5ec-4b4b7f393a4c None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:15.179027 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151d1dcc-4d03-47f1-b9b3-ff0e6dfeac55 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:15.207819 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 275/536] 10.222.0.85 () {56 vars in 1148 bytes} [Tue Jun 7 14:20:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:15.216513 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4453308-5bbf-47a6-a5ec-4b4b7f393a4c None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:15.249904 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36ae76f-57f1-4f7b-ad24-1144bf4648d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:15.252073 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36ae76f-57f1-4f7b-ad24-1144bf4648d6 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:15.253499 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36ae76f-57f1-4f7b-ad24-1144bf4648d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:15.253999 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36ae76f-57f1-4f7b-ad24-1144bf4648d6 None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:15.255881 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36ae76f-57f1-4f7b-ad24-1144bf4648d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:15.256822 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36ae76f-57f1-4f7b-ad24-1144bf4648d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:15.258332 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36ae76f-57f1-4f7b-ad24-1144bf4648d6 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:15.275274 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:15.286598 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 262/537] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jun 7 14:20:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:15.292618 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36ae76f-57f1-4f7b-ad24-1144bf4648d6 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:15.384765 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:15.405211 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4453308-5bbf-47a6-a5ec-4b4b7f393a4c None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:15.480460 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c899dcb-3f91-4845-ab10-5ee3e3e5d45d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:15.481062 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c899dcb-3f91-4845-ab10-5ee3e3e5d45d None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:15.481342 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c899dcb-3f91-4845-ab10-5ee3e3e5d45d None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:15.481622 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c899dcb-3f91-4845-ab10-5ee3e3e5d45d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:15.482648 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c899dcb-3f91-4845-ab10-5ee3e3e5d45d None admin] RBAC: Adding query filter params (role.id=17516844e8b947b389d62f50718b852d, scope.project.id=724785eb74e44daf8736fd01fc0d1839, user.id=b51da9867d9147f198b8077e953c4f9a) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:15.482962 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c899dcb-3f91-4845-ab10-5ee3e3e5d45d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:15.484494 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c899dcb-3f91-4845-ab10-5ee3e3e5d45d None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:15.496668 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36ae76f-57f1-4f7b-ad24-1144bf4648d6 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:15.502661 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 276/538] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jun 7 14:20:15 2022] GET /identity/v3/role_assignments?user.id=b51da9867d9147f198b8077e953c4f9a&scope.project.id=724785eb74e44daf8736fd01fc0d1839&role.id=17516844e8b947b389d62f50718b852d => generated 259 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:15.558935 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d848e3-4917-4956-827d-df45130f6e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:15.559745 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d848e3-4917-4956-827d-df45130f6e73 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:15.560030 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d848e3-4917-4956-827d-df45130f6e73 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:15.560304 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d848e3-4917-4956-827d-df45130f6e73 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:15.573691 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d848e3-4917-4956-827d-df45130f6e73 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:15.575181 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d848e3-4917-4956-827d-df45130f6e73 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:15.594877 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-45d848e3-4917-4956-827d-df45130f6e73 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:20:15.594877 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:15.594877 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:15.594877 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:15.594877 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:15.594877 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:15.594877 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:15.594877 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:15.594877 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:15.594877 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:15.594877 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:15.594877 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:15.594877 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 14:20:15.594877 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 14:20:15.594877 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 14:20:15.595799 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 14:20:15.595799 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:15.595799 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:15.595799 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:15.595799 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:15.595799 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:15.595799 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:15.595799 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:15.595799 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:15.595799 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:15.595799 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:15.595799 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:15.595799 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:15.595799 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:15.595799 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:15.595799 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:15.595799 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:15.597328 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 14:20:15.597328 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 14:20:15.597328 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 14:20:15.597328 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 14:20:15.597328 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 14:20:15.597328 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 14:20:15.597328 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:20:15.597328 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:15.597328 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 263/539] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jun 7 14:20:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:20:15.602082 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c899dcb-3f91-4845-ab10-5ee3e3e5d45d None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:15.761858 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:15.766952 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c899dcb-3f91-4845-ab10-5ee3e3e5d45d None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:15.804808 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbcfc89-c9d4-44cd-a54f-f03d61d41efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:15.805469 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbcfc89-c9d4-44cd-a54f-f03d61d41efa None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:15.805749 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbcfc89-c9d4-44cd-a54f-f03d61d41efa None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:15.821176 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbcfc89-c9d4-44cd-a54f-f03d61d41efa None admin] PATH_INFO: `/v3/projects` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:15.821176 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbcfc89-c9d4-44cd-a54f-f03d61d41efa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:15.821176 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbcfc89-c9d4-44cd-a54f-f03d61d41efa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:15.821176 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbcfc89-c9d4-44cd-a54f-f03d61d41efa None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:15.821176 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 277/540] 10.222.0.85 () {56 vars in 1148 bytes} [Tue Jun 7 14:20:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:15.826918 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d848e3-4917-4956-827d-df45130f6e73 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:15.853755 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e9b949-5435-4938-ba2f-8f6444d3c886 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:15.854062 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e9b949-5435-4938-ba2f-8f6444d3c886 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:15.854323 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e9b949-5435-4938-ba2f-8f6444d3c886 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:15.857684 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 278/541] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:20:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:20:15.863595 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f712c2f-1469-4d85-925c-b83cc4c7af7c None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:15.863890 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f712c2f-1469-4d85-925c-b83cc4c7af7c None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:15.864158 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f712c2f-1469-4d85-925c-b83cc4c7af7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:15.883642 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-9f712c2f-1469-4d85-925c-b83cc4c7af7c None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:20:15.885057 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-9f712c2f-1469-4d85-925c-b83cc4c7af7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:20:15.969238 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 279/542] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:20:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 14:20:15.987201 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d848e3-4917-4956-827d-df45130f6e73 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:15.993975 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f712c2f-1469-4d85-925c-b83cc4c7af7c None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:15.996359 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:16.049771 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f97699-f415-4711-ba82-1619dbf0c807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:16.050358 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f97699-f415-4711-ba82-1619dbf0c807 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:16.050646 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f97699-f415-4711-ba82-1619dbf0c807 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:16.050928 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f97699-f415-4711-ba82-1619dbf0c807 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:16.051780 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f97699-f415-4711-ba82-1619dbf0c807 None admin] RBAC: Adding query filter params (role.id=17516844e8b947b389d62f50718b852d, scope.project.id=724785eb74e44daf8736fd01fc0d1839, user.id=7ac4c85918c045b793a1a53300c989a4) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:16.052094 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f97699-f415-4711-ba82-1619dbf0c807 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:16.054352 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f97699-f415-4711-ba82-1619dbf0c807 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:16.066984 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 264/543] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jun 7 14:20:15 2022] GET /identity/v3/role_assignments?user.id=7ac4c85918c045b793a1a53300c989a4&scope.project.id=724785eb74e44daf8736fd01fc0d1839&role.id=17516844e8b947b389d62f50718b852d => generated 259 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:16.097702 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:16.119194 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f712c2f-1469-4d85-925c-b83cc4c7af7c None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:16.162768 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e046250-c5e1-4ce6-b625-a89bca9f9ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:16.164010 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e046250-c5e1-4ce6-b625-a89bca9f9ced None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:16.164535 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e046250-c5e1-4ce6-b625-a89bca9f9ced None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:16.165477 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e046250-c5e1-4ce6-b625-a89bca9f9ced None admin] PATH_INFO: `/v3/roles/service` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:16.178638 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e046250-c5e1-4ce6-b625-a89bca9f9ced None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:20:16.187820 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-5e046250-c5e1-4ce6-b625-a89bca9f9ced None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:20:16.187820 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:16.187820 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:16.187820 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:16.187820 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:16.187820 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:16.187820 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:16.187820 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:16.187820 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:16.187820 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:16.187820 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:16.187820 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:16.187820 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:20:16.187820 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:20:16.187820 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:20:16.187820 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:20:16.195836 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:20:16.195836 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:20:16.195836 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:20:16.195836 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:20:16.195836 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:16.195836 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:16.195836 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:16.195836 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:16.195836 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:16.195836 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:16.195836 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:16.195836 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:16.195836 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:16.195836 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:16.195836 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:16.195836 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:16.197094 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:16.197094 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:16.197094 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:16.197094 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:16.197094 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:20:16.197094 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:20:16.197094 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:20:16.197094 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:20:16.197094 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:20:16.197094 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:20:16.197094 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:20:16.197094 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:16.197094 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 280/544] 10.222.0.85 () {56 vars in 1102 bytes} [Tue Jun 7 14:20:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:16.201547 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f97699-f415-4711-ba82-1619dbf0c807 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:16.386234 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f97699-f415-4711-ba82-1619dbf0c807 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:16.427522 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb88de8d-b2e0-4e2b-88d9-529fe900a7e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:16.428198 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb88de8d-b2e0-4e2b-88d9-529fe900a7e9 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:16.428528 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb88de8d-b2e0-4e2b-88d9-529fe900a7e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:16.428881 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb88de8d-b2e0-4e2b-88d9-529fe900a7e9 None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:16.430139 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb88de8d-b2e0-4e2b-88d9-529fe900a7e9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:16.430468 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb88de8d-b2e0-4e2b-88d9-529fe900a7e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:16.431827 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb88de8d-b2e0-4e2b-88d9-529fe900a7e9 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:16.444887 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 265/545] 10.222.0.85 () {56 vars in 1103 bytes} [Tue Jun 7 14:20:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:16.453505 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e046250-c5e1-4ce6-b625-a89bca9f9ced None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:16.570650 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e046250-c5e1-4ce6-b625-a89bca9f9ced None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:16.607162 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c161e5f7-423f-4127-9949-586b66fbac06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:16.608340 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c161e5f7-423f-4127-9949-586b66fbac06 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:16.608856 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c161e5f7-423f-4127-9949-586b66fbac06 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:16.609322 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c161e5f7-423f-4127-9949-586b66fbac06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:16.621800 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c161e5f7-423f-4127-9949-586b66fbac06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:16.623605 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c161e5f7-423f-4127-9949-586b66fbac06 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:16.632332 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:16.649962 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-c161e5f7-423f-4127-9949-586b66fbac06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:16.649962 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:16.649962 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:16.649962 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:16.649962 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:16.649962 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:16.649962 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:16.649962 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:16.649962 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:16.649962 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:16.649962 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:16.649962 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:16.649962 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:20:16.649962 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:20:16.649962 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:20:16.650820 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:20:16.650820 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:16.650820 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:16.650820 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:16.650820 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:16.650820 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:16.650820 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:16.650820 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:16.650820 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:16.650820 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:16.650820 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:16.650820 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:16.650820 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:16.650820 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:16.650820 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:16.650820 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:16.650820 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:16.652154 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:20:16.652154 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:20:16.652154 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:16.652154 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:16.652154 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 281/546] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:20:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:16.652538 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb88de8d-b2e0-4e2b-88d9-529fe900a7e9 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:16.700742 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:16.795986 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb88de8d-b2e0-4e2b-88d9-529fe900a7e9 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:16.844005 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749cccc0-23ca-4483-b07b-0d93e25eced6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:16.844711 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749cccc0-23ca-4483-b07b-0d93e25eced6 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:16.844999 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749cccc0-23ca-4483-b07b-0d93e25eced6 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:16.845277 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749cccc0-23ca-4483-b07b-0d93e25eced6 None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:16.846020 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749cccc0-23ca-4483-b07b-0d93e25eced6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:16.846328 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749cccc0-23ca-4483-b07b-0d93e25eced6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:16.847718 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749cccc0-23ca-4483-b07b-0d93e25eced6 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:16.861785 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 266/547] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jun 7 14:20:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:16.867333 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c161e5f7-423f-4127-9949-586b66fbac06 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:17.141347 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c161e5f7-423f-4127-9949-586b66fbac06 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:17.186969 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693fbe5a-033f-4b88-8f5a-220531cfe4c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:17.197971 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693fbe5a-033f-4b88-8f5a-220531cfe4c3 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:17.197971 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693fbe5a-033f-4b88-8f5a-220531cfe4c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:17.197971 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693fbe5a-033f-4b88-8f5a-220531cfe4c3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:17.202798 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693fbe5a-033f-4b88-8f5a-220531cfe4c3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:17.204260 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693fbe5a-033f-4b88-8f5a-220531cfe4c3 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:17.232289 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-693fbe5a-033f-4b88-8f5a-220531cfe4c3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 07 14:20:17.232289 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:17.232289 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:17.232289 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:17.232289 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:17.232289 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:17.232289 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:17.232289 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:17.232289 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:17.232289 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:17.232289 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:17.232289 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:17.232289 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 14:20:17.232289 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 14:20:17.232289 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 14:20:17.232289 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 14:20:17.233233 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:17.233233 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:17.233233 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 14:20:17.233233 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:20:17.233233 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 14:20:17.233233 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:20:17.233233 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:17.233233 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:17.233233 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:17.233233 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:17.233233 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:17.233233 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:17.233233 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:17.233233 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:17.233233 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:17.233233 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:17.233233 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:17.233233 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:17.234121 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:17.234121 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:17.234121 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 14:20:17.234121 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 14:20:17.234121 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 14:20:17.234121 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 14:20:17.234121 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 14:20:17.234121 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 14:20:17.234121 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 07 14:20:17.234121 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:17.234121 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 282/548] 10.222.0.85 () {56 vars in 1096 bytes} [Tue Jun 7 14:20:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 362 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:17.237604 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749cccc0-23ca-4483-b07b-0d93e25eced6 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:17.278860 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74e061b-0f6f-4e39-adc8-90a755f4edef None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:17.279211 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74e061b-0f6f-4e39-adc8-90a755f4edef None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:17.279468 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74e061b-0f6f-4e39-adc8-90a755f4edef None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:17.281031 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 283/549] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:20:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 14:20:17.287034 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfe5337-ba25-46a8-b69d-4053454c0cc7 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:17.287328 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfe5337-ba25-46a8-b69d-4053454c0cc7 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:17.287606 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfe5337-ba25-46a8-b69d-4053454c0cc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:17.318263 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-8bfe5337-ba25-46a8-b69d-4053454c0cc7 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:20:17.319495 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-8bfe5337-ba25-46a8-b69d-4053454c0cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:20:17.408835 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749cccc0-23ca-4483-b07b-0d93e25eced6 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:17.410504 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:17.425200 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:17.427786 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 284/550] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:20:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:20:17.432411 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92370fa-929b-4ca0-b67a-24cf3d9318b4 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:17.432411 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92370fa-929b-4ca0-b67a-24cf3d9318b4 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:17.432411 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92370fa-929b-4ca0-b67a-24cf3d9318b4 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:17.432411 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 285/551] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:20:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:20:17.442289 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cc480c-fe1a-4a64-8a73-de7b281d03d3 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:17.442797 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cc480c-fe1a-4a64-8a73-de7b281d03d3 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:17.443256 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cc480c-fe1a-4a64-8a73-de7b281d03d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:17.466320 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-d8cc480c-fe1a-4a64-8a73-de7b281d03d3 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:20:17.466985 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a120e44-2e81-425b-a7c3-16069e5a653c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:17.467670 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a120e44-2e81-425b-a7c3-16069e5a653c None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:17.467960 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a120e44-2e81-425b-a7c3-16069e5a653c None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:17.468285 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a120e44-2e81-425b-a7c3-16069e5a653c None admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:17.468638 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-d8cc480c-fe1a-4a64-8a73-de7b281d03d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:20:17.469283 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a120e44-2e81-425b-a7c3-16069e5a653c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:17.469649 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a120e44-2e81-425b-a7c3-16069e5a653c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:17.471089 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a120e44-2e81-425b-a7c3-16069e5a653c None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:17.491647 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 267/552] 10.222.0.85 () {56 vars in 1135 bytes} [Tue Jun 7 14:20:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:17.492518 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a120e44-2e81-425b-a7c3-16069e5a653c None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:17.564538 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 286/553] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:20:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:20:17.564538 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cc480c-fe1a-4a64-8a73-de7b281d03d3 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:17.620687 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a120e44-2e81-425b-a7c3-16069e5a653c None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:17.658331 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf751a1-1ca3-43dd-9eae-509299151eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:17.659166 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf751a1-1ca3-43dd-9eae-509299151eb4 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:17.659456 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf751a1-1ca3-43dd-9eae-509299151eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:17.659729 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf751a1-1ca3-43dd-9eae-509299151eb4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:17.672280 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf751a1-1ca3-43dd-9eae-509299151eb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:20:17.682911 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-ccf751a1-1ca3-43dd-9eae-509299151eb4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:20:17.682911 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:17.682911 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:17.682911 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:17.682911 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:17.682911 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:17.682911 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:17.682911 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:17.682911 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:17.682911 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:17.682911 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:17.682911 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:17.682911 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:20:17.682911 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:20:17.682911 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:20:17.682911 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:20:17.683843 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:20:17.683843 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:20:17.683843 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:20:17.683843 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:20:17.683843 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:17.683843 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:17.683843 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:17.683843 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:17.683843 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:17.683843 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:17.683843 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:17.683843 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:17.683843 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:17.683843 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:17.683843 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:17.683843 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:17.684688 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:17.684688 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:17.684688 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:17.684688 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:17.684688 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:20:17.684688 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:20:17.684688 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:20:17.684688 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:20:17.684688 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:20:17.684688 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:20:17.684688 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:20:17.684688 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:17.690982 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 268/554] 10.222.0.85 () {56 vars in 1102 bytes} [Tue Jun 7 14:20:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:17.692033 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf751a1-1ca3-43dd-9eae-509299151eb4 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:17.702158 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cc480c-fe1a-4a64-8a73-de7b281d03d3 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:17.708773 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:17.724331 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:17.741593 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b5ce8e-2065-4816-aa0b-b17e4a31b1f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:17.742313 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b5ce8e-2065-4816-aa0b-b17e4a31b1f5 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:17.742598 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b5ce8e-2065-4816-aa0b-b17e4a31b1f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:17.742889 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b5ce8e-2065-4816-aa0b-b17e4a31b1f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:17.824572 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b5ce8e-2065-4816-aa0b-b17e4a31b1f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:17.826092 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b5ce8e-2065-4816-aa0b-b17e4a31b1f5 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:17.826988 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf751a1-1ca3-43dd-9eae-509299151eb4 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:17.878388 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9db2651-d76e-4b4e-95bb-c4bfbf91a2d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:17.879192 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9db2651-d76e-4b4e-95bb-c4bfbf91a2d5 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:17.879481 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9db2651-d76e-4b4e-95bb-c4bfbf91a2d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:17.879759 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9db2651-d76e-4b4e-95bb-c4bfbf91a2d5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:17.894712 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9db2651-d76e-4b4e-95bb-c4bfbf91a2d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:20:17.902618 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-e9db2651-d76e-4b4e-95bb-c4bfbf91a2d5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:20:17.902618 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:17.902618 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:17.902618 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:17.902618 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:17.902618 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:17.902618 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:17.902618 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:17.902618 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:17.902618 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:17.902618 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:17.902618 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:17.902618 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:20:17.902618 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:20:17.902618 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:20:17.902618 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:20:17.903518 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:20:17.903518 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:20:17.903518 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:20:17.903518 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:20:17.903518 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:17.903518 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:17.903518 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:17.903518 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:17.903518 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:17.903518 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:17.903518 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:17.903518 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:17.903518 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:17.903518 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:17.903518 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:17.903518 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:17.904334 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:17.904334 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:17.904334 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:17.904334 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:17.904334 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:20:17.904334 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:20:17.904334 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:20:17.904334 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:20:17.904334 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:20:17.904334 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:20:17.904334 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:20:17.904334 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:17.908746 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 269/555] 10.222.0.85 () {56 vars in 1102 bytes} [Tue Jun 7 14:20:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:17.909634 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9db2651-d76e-4b4e-95bb-c4bfbf91a2d5 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:17.962797 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 287/556] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jun 7 14:20:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 14:20:17.964266 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b5ce8e-2065-4816-aa0b-b17e4a31b1f5 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:17.989973 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:17.996450 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:18.032161 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9db2651-d76e-4b4e-95bb-c4bfbf91a2d5 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:18.080984 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b5ce8e-2065-4816-aa0b-b17e4a31b1f5 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:18.086171 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503c66f5-9c08-4029-9cdf-4414b7e55770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:18.086963 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c66f5-9c08-4029-9cdf-4414b7e55770 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:18.087456 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c66f5-9c08-4029-9cdf-4414b7e55770 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:18.087837 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c66f5-9c08-4029-9cdf-4414b7e55770 None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:18.088739 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503c66f5-9c08-4029-9cdf-4414b7e55770 None admin] RBAC: Adding query filter params (name=service) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:18.089156 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503c66f5-9c08-4029-9cdf-4414b7e55770 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:18.091233 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503c66f5-9c08-4029-9cdf-4414b7e55770 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:18.103556 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 270/557] 10.222.0.85 () {56 vars in 1103 bytes} [Tue Jun 7 14:20:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:18.105087 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503c66f5-9c08-4029-9cdf-4414b7e55770 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:18.122463 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34728ad-6cdb-4a26-8bb9-e5da9a85220a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:18.123206 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34728ad-6cdb-4a26-8bb9-e5da9a85220a None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:18.123493 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34728ad-6cdb-4a26-8bb9-e5da9a85220a None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:18.123768 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34728ad-6cdb-4a26-8bb9-e5da9a85220a None admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:18.124549 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34728ad-6cdb-4a26-8bb9-e5da9a85220a None admin] RBAC: Adding query filter params (name=service) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:18.124853 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34728ad-6cdb-4a26-8bb9-e5da9a85220a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:18.126287 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34728ad-6cdb-4a26-8bb9-e5da9a85220a None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:18.136583 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 288/558] 10.222.0.85 () {56 vars in 1103 bytes} [Tue Jun 7 14:20:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:18.136583 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f647b81-6a7a-4d8e-93f6-13c71e19f2de None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:18.136583 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f647b81-6a7a-4d8e-93f6-13c71e19f2de None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:18.136583 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f647b81-6a7a-4d8e-93f6-13c71e19f2de None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:18.137817 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 289/559] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:20:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:20:18.138519 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f647b81-6a7a-4d8e-93f6-13c71e19f2de None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:18.217702 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503c66f5-9c08-4029-9cdf-4414b7e55770 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:18.231654 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:18.250052 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:18.251919 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f647b81-6a7a-4d8e-93f6-13c71e19f2de None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:18.255168 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b865aca5-a851-4170-8fd5-24f511a5ccc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:18.255958 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b865aca5-a851-4170-8fd5-24f511a5ccc4 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:18.256241 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b865aca5-a851-4170-8fd5-24f511a5ccc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:18.256549 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b865aca5-a851-4170-8fd5-24f511a5ccc4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:18.267959 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b865aca5-a851-4170-8fd5-24f511a5ccc4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:18.269455 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b865aca5-a851-4170-8fd5-24f511a5ccc4 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:18.287200 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-b865aca5-a851-4170-8fd5-24f511a5ccc4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:18.287200 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:18.287200 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:18.287200 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:18.287200 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:18.287200 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:18.287200 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:18.287200 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:18.287200 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:18.287200 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:18.287200 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:18.287200 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:18.287200 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:20:18.287200 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:20:18.287200 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:20:18.288778 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:20:18.288778 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:18.288778 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:18.288778 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:18.288778 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:18.288778 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:18.288778 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:18.288778 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:18.288778 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:18.288778 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:18.288778 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:18.288778 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:18.288778 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:18.288778 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:18.288778 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:18.288778 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:18.288778 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:18.289668 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:20:18.289668 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:20:18.289668 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:18.289668 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:18.289668 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 271/560] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:20:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:18.292141 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9357e77c-7f3b-4f08-ab11-ebbdbf2a60c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:18.292975 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9357e77c-7f3b-4f08-ab11-ebbdbf2a60c7 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:18.293511 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9357e77c-7f3b-4f08-ab11-ebbdbf2a60c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:18.293677 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9357e77c-7f3b-4f08-ab11-ebbdbf2a60c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:18.294192 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b865aca5-a851-4170-8fd5-24f511a5ccc4 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:18.305152 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9357e77c-7f3b-4f08-ab11-ebbdbf2a60c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:18.306623 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9357e77c-7f3b-4f08-ab11-ebbdbf2a60c7 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:18.324674 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-9357e77c-7f3b-4f08-ab11-ebbdbf2a60c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:18.324674 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:18.324674 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:18.324674 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:18.324674 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:18.324674 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:18.324674 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:18.324674 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:18.324674 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:18.324674 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:18.324674 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:18.324674 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:18.324674 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:20:18.324674 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:20:18.324674 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:20:18.325686 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:20:18.325686 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:18.325686 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:18.325686 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:18.325686 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:18.325686 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:18.325686 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:18.325686 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:18.325686 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:18.325686 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:18.325686 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:18.325686 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:18.325686 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:18.325686 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:18.325686 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:18.325686 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:18.325686 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:18.326592 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:20:18.326592 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:20:18.326592 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:18.326592 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:18.326592 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 290/561] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:20:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:18.327388 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104686f1-a0c4-47e0-bfa9-bd76bf8e720c None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:18.327388 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104686f1-a0c4-47e0-bfa9-bd76bf8e720c None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:18.327666 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104686f1-a0c4-47e0-bfa9-bd76bf8e720c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:18.347472 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-104686f1-a0c4-47e0-bfa9-bd76bf8e720c None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:20:18.348689 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-104686f1-a0c4-47e0-bfa9-bd76bf8e720c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:20:18.412967 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b865aca5-a851-4170-8fd5-24f511a5ccc4 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:18.436437 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 291/562] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:20:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:20:18.436437 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104686f1-a0c4-47e0-bfa9-bd76bf8e720c None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:18.455854 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6f3278-aa4f-42f2-9656-244aadd44f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:18.455854 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f3278-aa4f-42f2-9656-244aadd44f1e None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:18.455854 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f3278-aa4f-42f2-9656-244aadd44f1e None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:18.455854 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f3278-aa4f-42f2-9656-244aadd44f1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:18.472171 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6f3278-aa4f-42f2-9656-244aadd44f1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:18.474350 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6f3278-aa4f-42f2-9656-244aadd44f1e None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:18.498317 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-7a6f3278-aa4f-42f2-9656-244aadd44f1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:18.498317 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:18.498317 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:18.498317 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:18.498317 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:18.498317 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:18.498317 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:18.498317 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:18.498317 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:18.498317 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:18.498317 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:18.498317 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:18.498317 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:20:18.498317 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:20:18.498317 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:20:18.499417 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:20:18.499417 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:18.499417 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:18.499417 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:18.499417 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:18.499417 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:18.499417 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:18.499417 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:18.499417 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:18.499417 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:18.499417 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:18.499417 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:18.499417 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:18.499417 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:18.499417 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:18.499417 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:18.499417 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:18.500509 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:20:18.500509 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:20:18.500509 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:18.500509 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:18.500509 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 272/563] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:20:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:18.501659 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6f3278-aa4f-42f2-9656-244aadd44f1e None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:18.515314 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:18.545329 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:18.558150 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104686f1-a0c4-47e0-bfa9-bd76bf8e720c None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:18.598050 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e12313-16f4-426a-b78b-0e27e78f3a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:18.598727 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e12313-16f4-426a-b78b-0e27e78f3a61 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:18.599017 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e12313-16f4-426a-b78b-0e27e78f3a61 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:18.599300 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e12313-16f4-426a-b78b-0e27e78f3a61 None admin] PATH_INFO: `/v3/domains` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:18.600031 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e12313-16f4-426a-b78b-0e27e78f3a61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:18.600345 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e12313-16f4-426a-b78b-0e27e78f3a61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:18.601871 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e12313-16f4-426a-b78b-0e27e78f3a61 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:18.613875 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 292/564] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jun 7 14:20:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:18.614699 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e12313-16f4-426a-b78b-0e27e78f3a61 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:18.623969 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6f3278-aa4f-42f2-9656-244aadd44f1e None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:18.660766 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8589676-fabf-43fc-9fd7-fa51b7489068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:18.661802 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8589676-fabf-43fc-9fd7-fa51b7489068 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:18.662375 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8589676-fabf-43fc-9fd7-fa51b7489068 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:18.662845 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8589676-fabf-43fc-9fd7-fa51b7489068 None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:18.663605 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8589676-fabf-43fc-9fd7-fa51b7489068 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:18.663918 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8589676-fabf-43fc-9fd7-fa51b7489068 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:18.665379 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8589676-fabf-43fc-9fd7-fa51b7489068 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:18.678097 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 273/565] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jun 7 14:20:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:18.678966 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8589676-fabf-43fc-9fd7-fa51b7489068 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:18.730274 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e12313-16f4-426a-b78b-0e27e78f3a61 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:18.755934 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:18.767442 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563c8886-745b-45d9-a2be-cf5e6c322ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:18.768311 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563c8886-745b-45d9-a2be-cf5e6c322ef8 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:18.768555 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563c8886-745b-45d9-a2be-cf5e6c322ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:18.768837 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563c8886-745b-45d9-a2be-cf5e6c322ef8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:18.779422 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:18.779902 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563c8886-745b-45d9-a2be-cf5e6c322ef8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:20:18.785139 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8589676-fabf-43fc-9fd7-fa51b7489068 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:18.787992 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-563c8886-745b-45d9-a2be-cf5e6c322ef8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:20:18.787992 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:18.787992 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:18.787992 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:18.787992 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:18.787992 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:18.787992 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:18.787992 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:18.787992 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:18.787992 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:18.787992 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:18.787992 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:18.787992 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:20:18.787992 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:20:18.787992 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:20:18.787992 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:20:18.788935 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:20:18.788935 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:20:18.788935 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:20:18.788935 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:20:18.788935 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:18.788935 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:18.788935 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:18.788935 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:18.788935 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:18.788935 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:18.788935 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:18.788935 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:18.788935 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:18.788935 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:18.788935 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:18.788935 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:18.789807 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:18.789807 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:18.789807 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:18.789807 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:18.789807 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:20:18.789807 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:20:18.789807 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:20:18.789807 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:20:18.789807 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:20:18.789807 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:20:18.789807 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:20:18.789807 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:18.789807 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 293/566] 10.222.0.85 () {56 vars in 1102 bytes} [Tue Jun 7 14:20:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:18.790491 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563c8886-745b-45d9-a2be-cf5e6c322ef8 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:18.825905 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87da9f1e-3d87-4e43-9bbb-4c241a866104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:18.826570 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87da9f1e-3d87-4e43-9bbb-4c241a866104 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:18.826858 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87da9f1e-3d87-4e43-9bbb-4c241a866104 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:18.827138 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87da9f1e-3d87-4e43-9bbb-4c241a866104 None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:18.828144 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87da9f1e-3d87-4e43-9bbb-4c241a866104 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:18.828144 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87da9f1e-3d87-4e43-9bbb-4c241a866104 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:18.829548 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87da9f1e-3d87-4e43-9bbb-4c241a866104 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:18.841544 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 274/567] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jun 7 14:20:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:18.842420 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87da9f1e-3d87-4e43-9bbb-4c241a866104 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:18.900552 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563c8886-745b-45d9-a2be-cf5e6c322ef8 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:18.935195 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3cb286-2539-4b9a-b958-5b1197fa806d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:18.935991 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3cb286-2539-4b9a-b958-5b1197fa806d None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:18.936282 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3cb286-2539-4b9a-b958-5b1197fa806d None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:18.936599 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3cb286-2539-4b9a-b958-5b1197fa806d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:18.943366 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87da9f1e-3d87-4e43-9bbb-4c241a866104 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:18.943541 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3cb286-2539-4b9a-b958-5b1197fa806d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:18.945586 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3cb286-2539-4b9a-b958-5b1197fa806d None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:18.959913 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-5f3cb286-2539-4b9a-b958-5b1197fa806d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:20:18.959913 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:18.959913 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:18.959913 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:18.959913 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:18.959913 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:18.959913 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:18.959913 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:18.959913 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:18.959913 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:18.959913 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:18.959913 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:18.959913 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 14:20:18.959913 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 14:20:18.959913 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 14:20:18.960889 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 14:20:18.960889 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:18.960889 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:18.960889 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:18.960889 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:18.960889 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:18.960889 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:18.960889 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:18.960889 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:18.960889 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:18.960889 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:18.960889 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:18.960889 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:18.960889 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:18.960889 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:18.960889 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:18.960889 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:18.966978 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 14:20:18.966978 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 14:20:18.966978 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 14:20:18.966978 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 14:20:18.966978 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 14:20:18.966978 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 14:20:18.966978 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:20:18.966978 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:18.966978 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 294/568] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jun 7 14:20:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:18.966978 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1992a06b-c8ca-4d6b-ac39-2f260b0081e1 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:18.966978 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1992a06b-c8ca-4d6b-ac39-2f260b0081e1 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:18.967691 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1992a06b-c8ca-4d6b-ac39-2f260b0081e1 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:18.967691 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 295/569] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:20:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:20:18.967691 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1992a06b-c8ca-4d6b-ac39-2f260b0081e1 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:18.982691 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9152731a-abe6-48ba-be1a-686a61a4f4f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:18.983482 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9152731a-abe6-48ba-be1a-686a61a4f4f8 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:18.983767 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9152731a-abe6-48ba-be1a-686a61a4f4f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:18.984038 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9152731a-abe6-48ba-be1a-686a61a4f4f8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:18.992480 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9152731a-abe6-48ba-be1a-686a61a4f4f8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:18.993913 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9152731a-abe6-48ba-be1a-686a61a4f4f8 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:19.008918 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-9152731a-abe6-48ba-be1a-686a61a4f4f8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 07 14:20:19.008918 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:19.008918 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:19.008918 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:19.008918 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:19.008918 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:19.008918 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:19.008918 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:19.008918 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:19.008918 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:19.008918 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:19.008918 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:19.008918 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 14:20:19.008918 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 14:20:19.008918 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 14:20:19.008918 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 14:20:19.009927 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:19.009927 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:19.009927 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 14:20:19.009927 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:20:19.009927 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 14:20:19.009927 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:20:19.009927 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:19.009927 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:19.009927 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:19.009927 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:19.009927 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:19.009927 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:19.009927 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:19.009927 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:19.009927 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:19.009927 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:19.009927 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:19.009927 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:19.010846 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:19.010846 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:19.010846 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 14:20:19.010846 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 14:20:19.010846 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 14:20:19.010846 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 14:20:19.010846 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 14:20:19.010846 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 14:20:19.010846 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 07 14:20:19.010846 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:19.010846 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 275/570] 10.222.0.85 () {56 vars in 1093 bytes} [Tue Jun 7 14:20:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:19.010846 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9152731a-abe6-48ba-be1a-686a61a4f4f8 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:19.015228 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:19.030371 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:19.072098 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1992a06b-c8ca-4d6b-ac39-2f260b0081e1 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:19.107240 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e8c7c6-1fbe-410b-9e7a-5b02581eee65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:19.107909 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e8c7c6-1fbe-410b-9e7a-5b02581eee65 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:19.108198 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e8c7c6-1fbe-410b-9e7a-5b02581eee65 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:19.108510 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e8c7c6-1fbe-410b-9e7a-5b02581eee65 None admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:19.109263 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e8c7c6-1fbe-410b-9e7a-5b02581eee65 None admin] RBAC: Adding query filter params (name=service) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:19.109661 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e8c7c6-1fbe-410b-9e7a-5b02581eee65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:19.111081 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e8c7c6-1fbe-410b-9e7a-5b02581eee65 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:19.115013 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9152731a-abe6-48ba-be1a-686a61a4f4f8 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:19.119036 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 296/571] 10.222.0.85 () {56 vars in 1103 bytes} [Tue Jun 7 14:20:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:19.119036 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e8c7c6-1fbe-410b-9e7a-5b02581eee65 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:19.148994 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf7f031-584d-4278-ab98-e1ffe6fe3e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:19.149785 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf7f031-584d-4278-ab98-e1ffe6fe3e87 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:19.150087 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf7f031-584d-4278-ab98-e1ffe6fe3e87 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:19.150348 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf7f031-584d-4278-ab98-e1ffe6fe3e87 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:19.157973 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf7f031-584d-4278-ab98-e1ffe6fe3e87 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:19.159400 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf7f031-584d-4278-ab98-e1ffe6fe3e87 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:19.174364 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-cbf7f031-584d-4278-ab98-e1ffe6fe3e87 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 07 14:20:19.174364 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:19.174364 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:19.174364 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:19.174364 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:19.174364 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:19.174364 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:19.174364 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:19.174364 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:19.174364 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:19.174364 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:19.174364 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:19.174364 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 14:20:19.174364 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 14:20:19.174364 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 14:20:19.174364 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 14:20:19.175620 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:19.175620 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:19.175620 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 14:20:19.175620 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:20:19.175620 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 14:20:19.175620 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:20:19.175620 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:19.175620 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:19.175620 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:19.175620 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:19.175620 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:19.175620 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:19.175620 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:19.175620 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:19.175620 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:19.175620 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:19.175620 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:19.175620 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:19.176611 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:19.176611 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:19.176611 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 14:20:19.176611 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 14:20:19.176611 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 14:20:19.176611 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 14:20:19.176611 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 14:20:19.176611 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 14:20:19.176611 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 07 14:20:19.176611 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:19.176611 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 276/572] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jun 7 14:20:19 2022] GET /identity/v3/users/cinder => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:19.177198 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3e1773-ced2-4bb3-ba9a-214cc18832c0 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:19.177335 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3e1773-ced2-4bb3-ba9a-214cc18832c0 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:19.177607 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3e1773-ced2-4bb3-ba9a-214cc18832c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:19.194530 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-2e3e1773-ced2-4bb3-ba9a-214cc18832c0 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:20:19.195709 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-2e3e1773-ced2-4bb3-ba9a-214cc18832c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:20:19.224090 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e8c7c6-1fbe-410b-9e7a-5b02581eee65 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:19.231665 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:19.259215 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97400504-9245-435c-b818-40419b41f455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:19.259804 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97400504-9245-435c-b818-40419b41f455 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:19.260088 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97400504-9245-435c-b818-40419b41f455 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:19.260388 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97400504-9245-435c-b818-40419b41f455 None admin] PATH_INFO: `/v3/projects` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:19.261183 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97400504-9245-435c-b818-40419b41f455 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:19.261492 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97400504-9245-435c-b818-40419b41f455 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:19.262920 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97400504-9245-435c-b818-40419b41f455 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:19.263950 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 277/573] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:20:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:20:19.264903 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3e1773-ced2-4bb3-ba9a-214cc18832c0 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:19.273153 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:19.293106 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 297/574] 10.222.0.85 () {56 vars in 1148 bytes} [Tue Jun 7 14:20:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:19.293106 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97400504-9245-435c-b818-40419b41f455 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:19.365987 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3e1773-ced2-4bb3-ba9a-214cc18832c0 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:19.395267 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97400504-9245-435c-b818-40419b41f455 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:19.404964 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ecc6b2-4957-4507-b7c0-3356992a4509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:19.405630 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ecc6b2-4957-4507-b7c0-3356992a4509 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:19.405917 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ecc6b2-4957-4507-b7c0-3356992a4509 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:19.406193 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ecc6b2-4957-4507-b7c0-3356992a4509 None admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:19.407035 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ecc6b2-4957-4507-b7c0-3356992a4509 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:19.407347 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ecc6b2-4957-4507-b7c0-3356992a4509 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:19.408942 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ecc6b2-4957-4507-b7c0-3356992a4509 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:19.426332 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 278/575] 10.222.0.85 () {56 vars in 1133 bytes} [Tue Jun 7 14:20:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:19.427139 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ecc6b2-4957-4507-b7c0-3356992a4509 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:19.430762 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeef10d-63f5-4519-a500-c20415d2273a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:19.432232 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeef10d-63f5-4519-a500-c20415d2273a None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:19.432566 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeef10d-63f5-4519-a500-c20415d2273a None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:19.432852 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeef10d-63f5-4519-a500-c20415d2273a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:19.444048 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aeef10d-63f5-4519-a500-c20415d2273a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:19.445569 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aeef10d-63f5-4519-a500-c20415d2273a None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:19.462620 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-0aeef10d-63f5-4519-a500-c20415d2273a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:19.462620 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:19.462620 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:19.462620 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:19.462620 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:19.462620 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:19.462620 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:19.462620 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:19.462620 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:19.462620 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:19.462620 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:19.462620 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:19.462620 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:20:19.462620 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:20:19.462620 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:20:19.463636 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:20:19.463636 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:19.463636 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:19.463636 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:19.463636 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:19.463636 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:19.463636 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:19.463636 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:19.463636 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:19.463636 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:19.463636 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:19.463636 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:19.463636 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:19.463636 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:19.463636 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:19.463636 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:19.463636 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:19.464578 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:20:19.464578 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:20:19.464578 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:19.464578 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:19.464578 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 298/576] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:20:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:19.465322 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeef10d-63f5-4519-a500-c20415d2273a None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:19.485650 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:19.488500 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:19.525702 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ecc6b2-4957-4507-b7c0-3356992a4509 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:19.560150 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5e7395-99ab-413d-a8cc-482ae5496f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:19.560849 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5e7395-99ab-413d-a8cc-482ae5496f30 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:19.561137 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5e7395-99ab-413d-a8cc-482ae5496f30 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:19.561421 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5e7395-99ab-413d-a8cc-482ae5496f30 None admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:19.562292 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5e7395-99ab-413d-a8cc-482ae5496f30 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:19.562606 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5e7395-99ab-413d-a8cc-482ae5496f30 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:19.564029 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5e7395-99ab-413d-a8cc-482ae5496f30 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:19.566886 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeef10d-63f5-4519-a500-c20415d2273a None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:19.581089 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 279/577] 10.222.0.85 () {56 vars in 1137 bytes} [Tue Jun 7 14:20:19 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:19.581897 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5e7395-99ab-413d-a8cc-482ae5496f30 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:19.601868 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84321d0-e109-4da1-89e2-77619a64cdb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:19.602717 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84321d0-e109-4da1-89e2-77619a64cdb2 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:19.603010 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84321d0-e109-4da1-89e2-77619a64cdb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:19.603298 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84321d0-e109-4da1-89e2-77619a64cdb2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:19.613372 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84321d0-e109-4da1-89e2-77619a64cdb2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:20:19.621936 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-c84321d0-e109-4da1-89e2-77619a64cdb2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:20:19.621936 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:19.621936 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:19.621936 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:19.621936 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:19.621936 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:19.621936 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:19.621936 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:19.621936 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:19.621936 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:19.621936 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:19.621936 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:19.621936 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:20:19.621936 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:20:19.621936 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:20:19.621936 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:20:19.623027 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:20:19.623027 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:20:19.623027 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:20:19.623027 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:20:19.623027 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:19.623027 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:19.623027 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:19.623027 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:19.623027 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:19.623027 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:19.623027 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:19.623027 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:19.623027 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:19.623027 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:19.623027 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:19.623027 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:19.623868 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:19.623868 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:19.623868 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:19.623868 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:19.623868 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:20:19.623868 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:20:19.623868 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:20:19.623868 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:20:19.623868 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:20:19.623868 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:20:19.623868 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:20:19.623868 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:19.623868 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 299/578] 10.222.0.85 () {56 vars in 1102 bytes} [Tue Jun 7 14:20:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:19.624583 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84321d0-e109-4da1-89e2-77619a64cdb2 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:19.688648 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5e7395-99ab-413d-a8cc-482ae5496f30 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:19.701729 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:19.711765 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:19.722654 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84321d0-e109-4da1-89e2-77619a64cdb2 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:19.723601 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1bf562-34e7-49e3-ac40-581344a9dada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:19.724397 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1bf562-34e7-49e3-ac40-581344a9dada None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:19.724701 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1bf562-34e7-49e3-ac40-581344a9dada None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:19.724985 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1bf562-34e7-49e3-ac40-581344a9dada None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:19.726377 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1bf562-34e7-49e3-ac40-581344a9dada None admin] RBAC: Adding query filter params (role.id=17516844e8b947b389d62f50718b852d, scope.project.id=724785eb74e44daf8736fd01fc0d1839, user.id=35aa67025f66459da0c0fc3cbfd45430) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:19.726697 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1bf562-34e7-49e3-ac40-581344a9dada None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:19.728131 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1bf562-34e7-49e3-ac40-581344a9dada None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:19.734379 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 280/579] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jun 7 14:20:19 2022] GET /identity/v3/role_assignments?user.id=35aa67025f66459da0c0fc3cbfd45430&scope.project.id=724785eb74e44daf8736fd01fc0d1839&role.id=17516844e8b947b389d62f50718b852d => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:19.735182 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1bf562-34e7-49e3-ac40-581344a9dada None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:19.765647 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeb90b7-0842-4844-b2e7-595962b6f3cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:19.766853 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeb90b7-0842-4844-b2e7-595962b6f3cc None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:19.767294 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeb90b7-0842-4844-b2e7-595962b6f3cc None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:19.767773 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeb90b7-0842-4844-b2e7-595962b6f3cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:19.781687 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeb90b7-0842-4844-b2e7-595962b6f3cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:19.783517 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeb90b7-0842-4844-b2e7-595962b6f3cc None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:19.804622 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-5eeb90b7-0842-4844-b2e7-595962b6f3cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:19.804622 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:19.804622 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:19.804622 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:19.804622 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:19.804622 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:19.804622 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:19.804622 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:19.804622 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:19.804622 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:19.804622 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:19.804622 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:19.804622 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:20:19.804622 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:20:19.804622 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:20:19.806284 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:20:19.806284 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:19.806284 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:19.806284 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:19.806284 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:19.806284 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:19.806284 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:19.806284 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:19.806284 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:19.806284 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:19.806284 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:19.806284 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:19.806284 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:19.806284 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:19.806284 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:19.806284 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:19.806284 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:19.807184 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:20:19.807184 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:20:19.807184 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:19.807184 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:19.807184 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 300/580] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:20:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:19.807468 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeb90b7-0842-4844-b2e7-595962b6f3cc None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:19.860925 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1bf562-34e7-49e3-ac40-581344a9dada None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:19.911465 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13af686-226f-4e0e-8435-b1143f5238d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:19.912149 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13af686-226f-4e0e-8435-b1143f5238d7 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:19.912466 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13af686-226f-4e0e-8435-b1143f5238d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:19.912739 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13af686-226f-4e0e-8435-b1143f5238d7 None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:19.914069 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13af686-226f-4e0e-8435-b1143f5238d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:19.914380 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13af686-226f-4e0e-8435-b1143f5238d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:19.915772 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13af686-226f-4e0e-8435-b1143f5238d7 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:19.932596 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 281/581] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jun 7 14:20:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:19.932596 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13af686-226f-4e0e-8435-b1143f5238d7 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:19.938824 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeb90b7-0842-4844-b2e7-595962b6f3cc None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:19.975117 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:19.993447 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:19.994344 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761d6e2a-1038-4945-b3a1-4b713fca88d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:19.995890 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761d6e2a-1038-4945-b3a1-4b713fca88d8 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:19.996362 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761d6e2a-1038-4945-b3a1-4b713fca88d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:19.996800 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761d6e2a-1038-4945-b3a1-4b713fca88d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:20.012580 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761d6e2a-1038-4945-b3a1-4b713fca88d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:20.014910 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761d6e2a-1038-4945-b3a1-4b713fca88d8 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:20.036007 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-761d6e2a-1038-4945-b3a1-4b713fca88d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:20.036007 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:20.036007 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:20.036007 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:20.036007 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:20.036007 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:20.036007 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:20.036007 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:20.036007 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:20.036007 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:20.036007 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:20.036007 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:20.036007 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:20:20.036007 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:20:20.036007 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:20:20.036980 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:20:20.036980 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:20.036980 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:20.036980 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:20.036980 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:20.036980 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:20.036980 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:20.036980 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:20.036980 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:20.036980 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:20.036980 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:20.036980 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:20.036980 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:20.036980 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:20.036980 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:20.036980 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:20.036980 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:20.037873 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:20:20.037873 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:20:20.037873 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:20.037873 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:20.039855 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 301/582] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:20:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:20.040846 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761d6e2a-1038-4945-b3a1-4b713fca88d8 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:20.069025 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13af686-226f-4e0e-8435-b1143f5238d7 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:20.112614 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dd04d7-d746-4057-b67e-f99d3240f360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:20.113650 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dd04d7-d746-4057-b67e-f99d3240f360 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:20.114085 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dd04d7-d746-4057-b67e-f99d3240f360 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:20.114498 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dd04d7-d746-4057-b67e-f99d3240f360 None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:20.115436 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dd04d7-d746-4057-b67e-f99d3240f360 None admin] RBAC: Adding query filter params (name=service) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:20.115958 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dd04d7-d746-4057-b67e-f99d3240f360 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:20.117671 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dd04d7-d746-4057-b67e-f99d3240f360 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:20.127383 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 282/583] 10.222.0.85 () {56 vars in 1103 bytes} [Tue Jun 7 14:20:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:20.133615 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dd04d7-d746-4057-b67e-f99d3240f360 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:20.166665 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761d6e2a-1038-4945-b3a1-4b713fca88d8 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:20.208262 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3defb2-8839-4618-82d5-3563c0100ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:20.208981 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3defb2-8839-4618-82d5-3563c0100ff3 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:20.209272 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3defb2-8839-4618-82d5-3563c0100ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:20.209553 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3defb2-8839-4618-82d5-3563c0100ff3 None admin] PATH_INFO: `/v3/domains` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:20.210854 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3defb2-8839-4618-82d5-3563c0100ff3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:20.211168 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3defb2-8839-4618-82d5-3563c0100ff3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:20.213036 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3defb2-8839-4618-82d5-3563c0100ff3 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:20.228887 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 302/584] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jun 7 14:20:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:20.229806 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3defb2-8839-4618-82d5-3563c0100ff3 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:20.272679 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dd04d7-d746-4057-b67e-f99d3240f360 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:20.281992 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:20.286570 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:20.316547 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aeec4a-ff88-4eef-a48f-94da2990cd60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:20.317438 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aeec4a-ff88-4eef-a48f-94da2990cd60 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:20.317724 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aeec4a-ff88-4eef-a48f-94da2990cd60 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:20.318002 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aeec4a-ff88-4eef-a48f-94da2990cd60 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:20.330004 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aeec4a-ff88-4eef-a48f-94da2990cd60 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:20.331444 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aeec4a-ff88-4eef-a48f-94da2990cd60 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:20.346391 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-f2aeec4a-ff88-4eef-a48f-94da2990cd60 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 07 14:20:20.346391 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:20.346391 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:20.346391 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:20.346391 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:20.346391 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:20.346391 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:20.346391 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:20.346391 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:20.346391 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:20.346391 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:20.346391 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:20.346391 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 14:20:20.346391 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 14:20:20.346391 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 14:20:20.346391 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 14:20:20.347553 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:20.347553 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:20.347553 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 14:20:20.347553 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:20:20.347553 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 14:20:20.347553 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:20:20.347553 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:20.347553 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:20.347553 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:20.347553 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:20.347553 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:20.347553 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:20.347553 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:20.347553 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:20.347553 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:20.347553 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:20.347553 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:20.347553 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:20.348669 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:20.348669 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:20.348669 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 14:20:20.348669 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 14:20:20.348669 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 14:20:20.348669 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 14:20:20.348669 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 14:20:20.348669 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 14:20:20.348669 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 07 14:20:20.348669 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:20.354255 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 283/585] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jun 7 14:20:20 2022] GET /identity/v3/users/glance => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:20.355122 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aeec4a-ff88-4eef-a48f-94da2990cd60 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:20.371844 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3defb2-8839-4618-82d5-3563c0100ff3 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:20.420475 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff2cc2f-c51b-4a40-9b87-834debb90760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:20.421153 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff2cc2f-c51b-4a40-9b87-834debb90760 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:20.421438 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff2cc2f-c51b-4a40-9b87-834debb90760 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:20.421717 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff2cc2f-c51b-4a40-9b87-834debb90760 None admin] PATH_INFO: `/v3/domains` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:20.422446 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff2cc2f-c51b-4a40-9b87-834debb90760 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:20.422754 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff2cc2f-c51b-4a40-9b87-834debb90760 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:20.424118 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff2cc2f-c51b-4a40-9b87-834debb90760 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:20.442080 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 303/586] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jun 7 14:20:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:20.442080 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff2cc2f-c51b-4a40-9b87-834debb90760 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:20.491467 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aeec4a-ff88-4eef-a48f-94da2990cd60 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:20.529258 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1941b4d3-2d89-492f-a4d7-2170d28140e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:20.530067 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1941b4d3-2d89-492f-a4d7-2170d28140e5 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:20.530351 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1941b4d3-2d89-492f-a4d7-2170d28140e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:20.530632 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1941b4d3-2d89-492f-a4d7-2170d28140e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:20.542799 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1941b4d3-2d89-492f-a4d7-2170d28140e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:20.544171 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1941b4d3-2d89-492f-a4d7-2170d28140e5 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:20.558146 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff2cc2f-c51b-4a40-9b87-834debb90760 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:20.559892 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:20.564822 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-1941b4d3-2d89-492f-a4d7-2170d28140e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:20.564822 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:20.564822 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:20.564822 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:20.564822 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:20.564822 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:20.564822 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:20.564822 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:20.564822 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:20.564822 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:20.564822 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:20.564822 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:20.564822 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:20:20.564822 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:20:20.564822 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:20:20.565705 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:20:20.565705 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:20.565705 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:20.565705 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:20.565705 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:20.565705 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:20.565705 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:20.565705 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:20.565705 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:20.565705 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:20.565705 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:20.565705 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:20.565705 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:20.565705 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:20.565705 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:20.565705 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:20.565705 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:20.566582 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:20:20.566582 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:20:20.566582 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:20.566582 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:20.566582 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 284/587] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:20:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:20.566850 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1941b4d3-2d89-492f-a4d7-2170d28140e5 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:20.592360 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:20.604783 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d19414-9162-4634-af8b-13ec1f8012fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:20.605583 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d19414-9162-4634-af8b-13ec1f8012fa None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:20.605868 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d19414-9162-4634-af8b-13ec1f8012fa None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:20.606152 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d19414-9162-4634-af8b-13ec1f8012fa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:20.613265 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d19414-9162-4634-af8b-13ec1f8012fa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:20.614716 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d19414-9162-4634-af8b-13ec1f8012fa None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:20.628684 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-d0d19414-9162-4634-af8b-13ec1f8012fa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:20:20.628684 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:20.628684 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:20.628684 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:20.628684 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:20.628684 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:20.628684 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:20.628684 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:20.628684 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:20.628684 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:20.628684 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:20.628684 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:20.628684 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 14:20:20.628684 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 14:20:20.628684 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 14:20:20.629575 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 14:20:20.629575 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:20.629575 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:20.629575 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:20.629575 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:20.629575 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:20.629575 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:20.629575 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:20.629575 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:20.629575 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:20.629575 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:20.629575 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:20.629575 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:20.629575 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:20.629575 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:20.629575 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:20.629575 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:20.630463 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 14:20:20.630463 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 14:20:20.630463 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 14:20:20.630463 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 14:20:20.630463 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 14:20:20.630463 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 14:20:20.630463 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:20:20.630463 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:20.630463 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 304/588] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jun 7 14:20:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:20.630953 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d19414-9162-4634-af8b-13ec1f8012fa None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:20.678074 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1941b4d3-2d89-492f-a4d7-2170d28140e5 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:20.712610 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fb4490-3085-4b93-b7c3-6ac624b41460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:20.713272 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fb4490-3085-4b93-b7c3-6ac624b41460 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:20.713558 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fb4490-3085-4b93-b7c3-6ac624b41460 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:20.713827 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fb4490-3085-4b93-b7c3-6ac624b41460 None admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:20.714659 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fb4490-3085-4b93-b7c3-6ac624b41460 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:20.714963 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fb4490-3085-4b93-b7c3-6ac624b41460 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:20.716392 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fb4490-3085-4b93-b7c3-6ac624b41460 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:20.739542 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 285/589] 10.222.0.85 () {56 vars in 1137 bytes} [Tue Jun 7 14:20:20 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:20.739542 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fb4490-3085-4b93-b7c3-6ac624b41460 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:20.745268 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d19414-9162-4634-af8b-13ec1f8012fa None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:20.783012 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5703f7-1081-4d45-ab39-ab86bcdd990c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:20.783804 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5703f7-1081-4d45-ab39-ab86bcdd990c None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:20.784088 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5703f7-1081-4d45-ab39-ab86bcdd990c None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:20.784361 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5703f7-1081-4d45-ab39-ab86bcdd990c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:20.792499 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5703f7-1081-4d45-ab39-ab86bcdd990c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:20.794032 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5703f7-1081-4d45-ab39-ab86bcdd990c None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:20.807819 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-3a5703f7-1081-4d45-ab39-ab86bcdd990c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:20:20.807819 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:20.807819 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:20.807819 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:20.807819 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:20.807819 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:20.807819 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:20.807819 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:20.807819 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:20.807819 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:20.807819 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:20.807819 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:20.807819 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 14:20:20.807819 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 14:20:20.807819 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 14:20:20.808941 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 14:20:20.808941 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:20.808941 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:20.808941 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:20.808941 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:20.808941 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:20.808941 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:20.808941 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:20.808941 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:20.808941 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:20.808941 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:20.808941 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:20.808941 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:20.808941 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:20.808941 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:20.808941 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:20.808941 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:20.809854 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 14:20:20.809854 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 14:20:20.809854 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 14:20:20.809854 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 14:20:20.809854 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 14:20:20.809854 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 14:20:20.809854 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:20:20.809854 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:20.809854 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 305/590] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jun 7 14:20:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:20.809854 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5703f7-1081-4d45-ab39-ab86bcdd990c None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:20.824893 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:20.837647 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:20.846773 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fb4490-3085-4b93-b7c3-6ac624b41460 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:20.885155 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327eff91-9d7d-44c7-90fb-250295bef8d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:20.885818 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327eff91-9d7d-44c7-90fb-250295bef8d5 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:20.886096 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327eff91-9d7d-44c7-90fb-250295bef8d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:20.886366 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327eff91-9d7d-44c7-90fb-250295bef8d5 None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:20.887088 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327eff91-9d7d-44c7-90fb-250295bef8d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:20.887394 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327eff91-9d7d-44c7-90fb-250295bef8d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:20.888782 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327eff91-9d7d-44c7-90fb-250295bef8d5 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:20.901416 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 286/591] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jun 7 14:20:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:20.902856 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327eff91-9d7d-44c7-90fb-250295bef8d5 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:20.924983 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5703f7-1081-4d45-ab39-ab86bcdd990c None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:20.974595 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c130ab-9004-4e43-bd77-1ac6724ddf65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:20.975651 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c130ab-9004-4e43-bd77-1ac6724ddf65 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:20.976207 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c130ab-9004-4e43-bd77-1ac6724ddf65 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:20.976759 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c130ab-9004-4e43-bd77-1ac6724ddf65 None admin] PATH_INFO: `/v3/projects` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:20.977673 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c130ab-9004-4e43-bd77-1ac6724ddf65 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:20.978092 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c130ab-9004-4e43-bd77-1ac6724ddf65 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:20.979686 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c130ab-9004-4e43-bd77-1ac6724ddf65 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:20.996582 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 306/592] 10.222.0.85 () {56 vars in 1148 bytes} [Tue Jun 7 14:20:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:20.996582 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c130ab-9004-4e43-bd77-1ac6724ddf65 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:21.022320 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327eff91-9d7d-44c7-90fb-250295bef8d5 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:21.060455 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac3e918-ad7a-4d3f-baf7-0b8253a0d423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:21.061192 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac3e918-ad7a-4d3f-baf7-0b8253a0d423 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:21.061590 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac3e918-ad7a-4d3f-baf7-0b8253a0d423 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:21.061872 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac3e918-ad7a-4d3f-baf7-0b8253a0d423 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:21.068779 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:21.073862 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac3e918-ad7a-4d3f-baf7-0b8253a0d423 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:21.075312 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac3e918-ad7a-4d3f-baf7-0b8253a0d423 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:21.079426 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:21.094523 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-eac3e918-ad7a-4d3f-baf7-0b8253a0d423 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:21.094523 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:21.094523 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:21.094523 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:21.094523 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:21.094523 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:21.094523 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:21.094523 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:21.094523 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:21.094523 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:21.094523 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:21.094523 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:21.094523 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:20:21.094523 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:20:21.094523 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:20:21.095415 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:20:21.095415 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:21.095415 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:21.095415 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:21.095415 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:21.095415 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:21.095415 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:21.095415 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:21.095415 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:21.095415 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:21.095415 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:21.095415 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:21.095415 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:21.095415 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:21.095415 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:21.095415 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:21.095415 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:21.096294 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:20:21.096294 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:20:21.096294 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:21.096294 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:21.096294 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 287/593] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:20:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:21.096998 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac3e918-ad7a-4d3f-baf7-0b8253a0d423 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:21.103972 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c130ab-9004-4e43-bd77-1ac6724ddf65 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:21.138987 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05579610-487e-4233-9b9d-862f6fc4b67a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:21.139702 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05579610-487e-4233-9b9d-862f6fc4b67a None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:21.139987 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05579610-487e-4233-9b9d-862f6fc4b67a None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:21.140266 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05579610-487e-4233-9b9d-862f6fc4b67a None admin] PATH_INFO: `/v3/projects` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:21.141101 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05579610-487e-4233-9b9d-862f6fc4b67a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:21.141417 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05579610-487e-4233-9b9d-862f6fc4b67a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:21.142841 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05579610-487e-4233-9b9d-862f6fc4b67a None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:21.156296 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 307/594] 10.222.0.85 () {56 vars in 1148 bytes} [Tue Jun 7 14:20:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:21.157116 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05579610-487e-4233-9b9d-862f6fc4b67a None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:21.209114 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac3e918-ad7a-4d3f-baf7-0b8253a0d423 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:21.251465 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108d2988-1c0a-4158-8e69-2213bfaf28d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:21.251465 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108d2988-1c0a-4158-8e69-2213bfaf28d7 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:21.251465 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108d2988-1c0a-4158-8e69-2213bfaf28d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:21.251465 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108d2988-1c0a-4158-8e69-2213bfaf28d7 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:21.264239 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108d2988-1c0a-4158-8e69-2213bfaf28d7 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:21.265721 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108d2988-1c0a-4158-8e69-2213bfaf28d7 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:21.275836 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05579610-487e-4233-9b9d-862f6fc4b67a None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:21.281662 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-108d2988-1c0a-4158-8e69-2213bfaf28d7 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 07 14:20:21.281662 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:21.281662 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:21.281662 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:21.281662 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:21.281662 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:21.281662 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:21.281662 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:21.281662 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:21.281662 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:21.281662 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:21.281662 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:21.281662 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 14:20:21.281662 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 14:20:21.281662 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 14:20:21.281662 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 14:20:21.282942 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:21.282942 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:21.282942 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 14:20:21.282942 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:20:21.282942 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 14:20:21.282942 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:20:21.282942 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:21.282942 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:21.282942 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:21.282942 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:21.282942 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:21.282942 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:21.282942 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:21.282942 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:21.282942 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:21.282942 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:21.282942 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:21.282942 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:21.283850 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:21.283850 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:21.283850 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 14:20:21.283850 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 14:20:21.283850 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 14:20:21.283850 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 14:20:21.283850 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 14:20:21.283850 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 14:20:21.283850 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 07 14:20:21.283850 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:21.283850 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 288/595] 10.222.0.85 () {56 vars in 1102 bytes} [Tue Jun 7 14:20:21 2022] GET /identity/v3/users/neutron => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:21.283850 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108d2988-1c0a-4158-8e69-2213bfaf28d7 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:21.316357 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c493fe50-7237-4287-afb1-d0a58e09bc8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:21.317199 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c493fe50-7237-4287-afb1-d0a58e09bc8e None admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:21.317487 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c493fe50-7237-4287-afb1-d0a58e09bc8e None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:21.317765 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c493fe50-7237-4287-afb1-d0a58e09bc8e None admin] PATH_INFO: `/v3/projects/724785eb74e44daf8736fd01fc0d1839/users/8e857db4902e42989d0d040bbc8517c8/roles/17516844e8b947b389d62f50718b852d` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:21.330917 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:21.334007 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c493fe50-7237-4287-afb1-d0a58e09bc8e None admin] RBAC: Authorizing `identity:create_grant(project_id=724785eb74e44daf8736fd01fc0d1839, user_id=8e857db4902e42989d0d040bbc8517c8, role_id=17516844e8b947b389d62f50718b852d)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:21.335498 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c493fe50-7237-4287-afb1-d0a58e09bc8e None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:21.340001 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:21.390379 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108d2988-1c0a-4158-8e69-2213bfaf28d7 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:21.424020 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9c4fb7-5df9-47cb-812b-f372d66a8481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:21.424760 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9c4fb7-5df9-47cb-812b-f372d66a8481 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:21.425049 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9c4fb7-5df9-47cb-812b-f372d66a8481 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:21.425331 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9c4fb7-5df9-47cb-812b-f372d66a8481 None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:21.426052 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9c4fb7-5df9-47cb-812b-f372d66a8481 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:21.426362 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9c4fb7-5df9-47cb-812b-f372d66a8481 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:21.427714 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9c4fb7-5df9-47cb-812b-f372d66a8481 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:21.444224 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 289/596] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jun 7 14:20:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:21.444224 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9c4fb7-5df9-47cb-812b-f372d66a8481 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:21.449207 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 308/597] 10.222.0.85 () {58 vars in 1439 bytes} [Tue Jun 7 14:20:21 2022] PUT /identity/v3/projects/724785eb74e44daf8736fd01fc0d1839/users/8e857db4902e42989d0d040bbc8517c8/roles/17516844e8b947b389d62f50718b852d => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:20:21.456565 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c493fe50-7237-4287-afb1-d0a58e09bc8e None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:21.583655 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9c4fb7-5df9-47cb-812b-f372d66a8481 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:21.596319 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:21.602308 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c493fe50-7237-4287-afb1-d0a58e09bc8e None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:21.628164 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5dfacf-1949-4d9f-90f6-85898797b48a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:21.629106 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5dfacf-1949-4d9f-90f6-85898797b48a None admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:21.629391 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5dfacf-1949-4d9f-90f6-85898797b48a None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:21.629667 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5dfacf-1949-4d9f-90f6-85898797b48a None admin] PATH_INFO: `/v3/projects/724785eb74e44daf8736fd01fc0d1839/users/89d81f597cb44c56a966537089bcce4e/roles/17516844e8b947b389d62f50718b852d` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:21.652581 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5dfacf-1949-4d9f-90f6-85898797b48a None admin] RBAC: Authorizing `identity:create_grant(project_id=724785eb74e44daf8736fd01fc0d1839, user_id=89d81f597cb44c56a966537089bcce4e, role_id=17516844e8b947b389d62f50718b852d)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:21.654195 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1046bfe1-521f-4923-adf3-edab71d99f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:21.654876 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5dfacf-1949-4d9f-90f6-85898797b48a None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:21.655493 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1046bfe1-521f-4923-adf3-edab71d99f7f None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:21.655781 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1046bfe1-521f-4923-adf3-edab71d99f7f None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:21.656852 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1046bfe1-521f-4923-adf3-edab71d99f7f None admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:21.657716 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1046bfe1-521f-4923-adf3-edab71d99f7f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:21.658029 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1046bfe1-521f-4923-adf3-edab71d99f7f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:21.659460 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1046bfe1-521f-4923-adf3-edab71d99f7f None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:21.682568 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 309/598] 10.222.0.85 () {56 vars in 1139 bytes} [Tue Jun 7 14:20:21 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:21.683454 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1046bfe1-521f-4923-adf3-edab71d99f7f None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:21.736396 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 290/599] 10.222.0.85 () {58 vars in 1439 bytes} [Tue Jun 7 14:20:21 2022] PUT /identity/v3/projects/724785eb74e44daf8736fd01fc0d1839/users/89d81f597cb44c56a966537089bcce4e/roles/17516844e8b947b389d62f50718b852d => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:20:21.736396 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5dfacf-1949-4d9f-90f6-85898797b48a None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:21.782971 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:21.846994 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1046bfe1-521f-4923-adf3-edab71d99f7f None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:21.891415 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9694e544-5b89-487c-b771-14c9b9ccb5a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:21.892231 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9694e544-5b89-487c-b771-14c9b9ccb5a5 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:21.892557 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9694e544-5b89-487c-b771-14c9b9ccb5a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:21.892856 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9694e544-5b89-487c-b771-14c9b9ccb5a5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:21.902312 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9694e544-5b89-487c-b771-14c9b9ccb5a5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:21.904586 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9694e544-5b89-487c-b771-14c9b9ccb5a5 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:21.926737 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-9694e544-5b89-487c-b771-14c9b9ccb5a5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:20:21.926737 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:21.926737 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:21.926737 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:21.926737 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:21.926737 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:21.926737 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:21.926737 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:21.926737 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:21.926737 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:21.926737 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:21.926737 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:21.926737 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 14:20:21.926737 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 14:20:21.926737 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 14:20:21.927700 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 14:20:21.927700 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:21.927700 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:21.927700 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:21.927700 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:21.927700 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:21.927700 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:21.927700 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:21.927700 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:21.927700 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:21.927700 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:21.927700 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:21.927700 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:21.927700 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:21.927700 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:21.927700 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:21.927700 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:21.928619 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 14:20:21.928619 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 14:20:21.928619 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 14:20:21.928619 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 14:20:21.928619 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 14:20:21.928619 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 14:20:21.928619 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:20:21.928619 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:21.932464 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 310/600] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jun 7 14:20:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:21.940186 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9694e544-5b89-487c-b771-14c9b9ccb5a5 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:21.990986 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5dfacf-1949-4d9f-90f6-85898797b48a None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:22.035057 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67596f3-0be5-4ca3-a64f-84f692afaaf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:22.036041 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67596f3-0be5-4ca3-a64f-84f692afaaf4 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:22.036304 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67596f3-0be5-4ca3-a64f-84f692afaaf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:22.036613 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67596f3-0be5-4ca3-a64f-84f692afaaf4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:22.049306 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67596f3-0be5-4ca3-a64f-84f692afaaf4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:22.050874 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67596f3-0be5-4ca3-a64f-84f692afaaf4 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:22.073682 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-c67596f3-0be5-4ca3-a64f-84f692afaaf4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:22.073682 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:22.073682 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:22.073682 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:22.073682 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:22.073682 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:22.073682 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:22.073682 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:22.073682 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:22.073682 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:22.073682 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:22.073682 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:22.073682 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:20:22.073682 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:20:22.073682 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:20:22.074644 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:20:22.074644 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:22.074644 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:22.074644 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:22.074644 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:22.074644 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:22.074644 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:22.074644 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:22.074644 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:22.074644 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:22.074644 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:22.074644 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:22.074644 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:22.074644 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:22.074644 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:22.074644 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:22.074644 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:22.076171 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:20:22.076171 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:20:22.076171 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:22.076171 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:22.076171 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 291/601] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:20:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:22.076689 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c419ac5-50ba-49c8-8bcc-e6d25ab8e99a None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:22.076976 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c419ac5-50ba-49c8-8bcc-e6d25ab8e99a None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:22.077252 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c419ac5-50ba-49c8-8bcc-e6d25ab8e99a None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:22.078746 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 292/602] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:20:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:20:22.083668 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c419ac5-50ba-49c8-8bcc-e6d25ab8e99a None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:22.095150 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9694e544-5b89-487c-b771-14c9b9ccb5a5 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:22.102178 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:22.111731 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:22.142767 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ad64c6-aeb5-4264-bd23-969a31b6d479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:22.143772 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ad64c6-aeb5-4264-bd23-969a31b6d479 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:22.144242 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ad64c6-aeb5-4264-bd23-969a31b6d479 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:22.145149 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ad64c6-aeb5-4264-bd23-969a31b6d479 None admin] PATH_INFO: `/v3/projects` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:22.146399 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ad64c6-aeb5-4264-bd23-969a31b6d479 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:22.146880 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ad64c6-aeb5-4264-bd23-969a31b6d479 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:22.149902 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ad64c6-aeb5-4264-bd23-969a31b6d479 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:22.165986 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 311/603] 10.222.0.85 () {56 vars in 1148 bytes} [Tue Jun 7 14:20:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:22.167092 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991a9a37-7911-43a3-8c08-4a226969efc7 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:22.167382 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991a9a37-7911-43a3-8c08-4a226969efc7 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:22.167945 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991a9a37-7911-43a3-8c08-4a226969efc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:22.192625 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-991a9a37-7911-43a3-8c08-4a226969efc7 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:20:22.193803 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-991a9a37-7911-43a3-8c08-4a226969efc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:20:22.209204 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c419ac5-50ba-49c8-8bcc-e6d25ab8e99a None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:22.253395 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0514b4af-d4f3-42da-b586-685a026f07f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:22.254080 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0514b4af-d4f3-42da-b586-685a026f07f7 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:22.254376 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0514b4af-d4f3-42da-b586-685a026f07f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:22.254618 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0514b4af-d4f3-42da-b586-685a026f07f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:22.255425 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0514b4af-d4f3-42da-b586-685a026f07f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:22.255729 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0514b4af-d4f3-42da-b586-685a026f07f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:22.257785 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0514b4af-d4f3-42da-b586-685a026f07f7 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:22.271028 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 293/604] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jun 7 14:20:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:22.271841 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0514b4af-d4f3-42da-b586-685a026f07f7 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:22.288198 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 312/605] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:20:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:20:22.289707 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991a9a37-7911-43a3-8c08-4a226969efc7 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:22.383624 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:22.404841 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:22.408136 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991a9a37-7911-43a3-8c08-4a226969efc7 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:22.426409 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0514b4af-d4f3-42da-b586-685a026f07f7 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:22.446915 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cf5bd7-43e7-41c3-a65d-b264a6903caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:22.447877 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cf5bd7-43e7-41c3-a65d-b264a6903caa None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:22.448324 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cf5bd7-43e7-41c3-a65d-b264a6903caa None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:22.448677 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cf5bd7-43e7-41c3-a65d-b264a6903caa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:22.457203 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cf5bd7-43e7-41c3-a65d-b264a6903caa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:22.458914 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cf5bd7-43e7-41c3-a65d-b264a6903caa None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:22.472043 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5ece1e-638f-44d5-bb22-017b70ab5b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:22.473197 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5ece1e-638f-44d5-bb22-017b70ab5b18 None admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:22.473629 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5ece1e-638f-44d5-bb22-017b70ab5b18 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:22.474054 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5ece1e-638f-44d5-bb22-017b70ab5b18 None admin] PATH_INFO: `/v3/projects/724785eb74e44daf8736fd01fc0d1839/users/b51da9867d9147f198b8077e953c4f9a/roles/17516844e8b947b389d62f50718b852d` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:22.475269 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-c1cf5bd7-43e7-41c3-a65d-b264a6903caa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:20:22.475269 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:22.475269 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:22.475269 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:22.475269 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:22.475269 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:22.475269 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:22.475269 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:22.475269 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:22.475269 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:22.475269 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:22.475269 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:22.475269 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 14:20:22.475269 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 14:20:22.475269 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 14:20:22.476417 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 14:20:22.476417 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:22.476417 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:22.476417 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:22.476417 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:22.476417 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:22.476417 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:22.476417 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:22.476417 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:22.476417 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:22.476417 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:22.476417 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:22.476417 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:22.476417 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:22.476417 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:22.476417 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:22.476417 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:22.477553 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 14:20:22.477553 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 14:20:22.477553 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 14:20:22.477553 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 14:20:22.477553 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 14:20:22.477553 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 14:20:22.477553 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:20:22.477553 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:22.479268 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 313/606] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jun 7 14:20:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:22.480314 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cf5bd7-43e7-41c3-a65d-b264a6903caa None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:22.499156 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5ece1e-638f-44d5-bb22-017b70ab5b18 None admin] RBAC: Authorizing `identity:create_grant(project_id=724785eb74e44daf8736fd01fc0d1839, user_id=b51da9867d9147f198b8077e953c4f9a, role_id=17516844e8b947b389d62f50718b852d)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:22.501627 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5ece1e-638f-44d5-bb22-017b70ab5b18 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:22.596655 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cf5bd7-43e7-41c3-a65d-b264a6903caa None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:22.636828 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 294/607] 10.222.0.85 () {58 vars in 1439 bytes} [Tue Jun 7 14:20:22 2022] PUT /identity/v3/projects/724785eb74e44daf8736fd01fc0d1839/users/b51da9867d9147f198b8077e953c4f9a/roles/17516844e8b947b389d62f50718b852d => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:20:22.638085 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5ece1e-638f-44d5-bb22-017b70ab5b18 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:22.656942 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a422a3-64a4-45c6-8d9e-0729cf70a963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:22.658006 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a422a3-64a4-45c6-8d9e-0729cf70a963 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:22.660606 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a422a3-64a4-45c6-8d9e-0729cf70a963 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:22.661106 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a422a3-64a4-45c6-8d9e-0729cf70a963 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:22.692767 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a422a3-64a4-45c6-8d9e-0729cf70a963 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:20:22.709313 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-48a422a3-64a4-45c6-8d9e-0729cf70a963 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:20:22.709313 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:22.709313 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:22.709313 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:22.709313 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:22.709313 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:22.709313 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:22.709313 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:22.709313 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:22.709313 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:22.709313 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:22.709313 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:22.709313 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:20:22.709313 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:20:22.709313 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:20:22.709313 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:20:22.711031 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:20:22.711031 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:20:22.711031 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:20:22.711031 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:20:22.711031 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:22.711031 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:22.711031 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:22.711031 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:22.711031 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:22.711031 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:22.711031 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:22.711031 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:22.711031 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:22.711031 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:22.711031 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:22.711031 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:22.712031 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:22.712031 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:22.712031 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:22.712031 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:22.712031 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:20:22.712031 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:20:22.712031 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:20:22.712031 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:20:22.712031 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:20:22.712031 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:20:22.712031 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:20:22.712031 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:22.716736 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 314/608] 10.222.0.85 () {56 vars in 1102 bytes} [Tue Jun 7 14:20:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:22.720394 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a422a3-64a4-45c6-8d9e-0729cf70a963 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:22.771518 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:22.833070 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5ece1e-638f-44d5-bb22-017b70ab5b18 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:22.873267 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:22.891979 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddefaf0-8d85-4fdf-ad6c-8c2f687eb4c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:22.894599 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddefaf0-8d85-4fdf-ad6c-8c2f687eb4c8 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:22.894599 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddefaf0-8d85-4fdf-ad6c-8c2f687eb4c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:22.894599 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddefaf0-8d85-4fdf-ad6c-8c2f687eb4c8 None admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:22.894599 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddefaf0-8d85-4fdf-ad6c-8c2f687eb4c8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:22.894599 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddefaf0-8d85-4fdf-ad6c-8c2f687eb4c8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:22.895909 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddefaf0-8d85-4fdf-ad6c-8c2f687eb4c8 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:22.913039 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 295/609] 10.222.0.85 () {56 vars in 1148 bytes} [Tue Jun 7 14:20:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:22.918939 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddefaf0-8d85-4fdf-ad6c-8c2f687eb4c8 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:22.932051 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a422a3-64a4-45c6-8d9e-0729cf70a963 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:22.983448 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec208c3-6950-47c1-b942-58a38c714ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:22.985107 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec208c3-6950-47c1-b942-58a38c714ad0 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:22.986493 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec208c3-6950-47c1-b942-58a38c714ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:22.986953 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec208c3-6950-47c1-b942-58a38c714ad0 None admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:22.988900 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec208c3-6950-47c1-b942-58a38c714ad0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:22.990732 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec208c3-6950-47c1-b942-58a38c714ad0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:22.993453 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec208c3-6950-47c1-b942-58a38c714ad0 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:23.013975 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 315/610] 10.222.0.85 () {56 vars in 1103 bytes} [Tue Jun 7 14:20:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:23.018777 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec208c3-6950-47c1-b942-58a38c714ad0 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:23.103197 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddefaf0-8d85-4fdf-ad6c-8c2f687eb4c8 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:23.174723 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404d71ea-79d2-4ca0-a04c-84b085af59e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:23.175856 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404d71ea-79d2-4ca0-a04c-84b085af59e6 None admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:23.177594 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404d71ea-79d2-4ca0-a04c-84b085af59e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:23.178059 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404d71ea-79d2-4ca0-a04c-84b085af59e6 None admin] PATH_INFO: `/v3/projects/724785eb74e44daf8736fd01fc0d1839/users/7ac4c85918c045b793a1a53300c989a4/roles/17516844e8b947b389d62f50718b852d` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:23.185156 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:23.204525 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404d71ea-79d2-4ca0-a04c-84b085af59e6 None admin] RBAC: Authorizing `identity:create_grant(project_id=724785eb74e44daf8736fd01fc0d1839, user_id=7ac4c85918c045b793a1a53300c989a4, role_id=17516844e8b947b389d62f50718b852d)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:23.204525 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404d71ea-79d2-4ca0-a04c-84b085af59e6 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:23.214665 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec208c3-6950-47c1-b942-58a38c714ad0 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:23.250622 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec3fedb-ea58-4f7f-b374-026f06f77bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:23.251481 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec3fedb-ea58-4f7f-b374-026f06f77bed None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:23.251775 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec3fedb-ea58-4f7f-b374-026f06f77bed None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:23.252060 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec3fedb-ea58-4f7f-b374-026f06f77bed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:23.263656 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec3fedb-ea58-4f7f-b374-026f06f77bed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:23.265201 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec3fedb-ea58-4f7f-b374-026f06f77bed None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:23.284759 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-1ec3fedb-ea58-4f7f-b374-026f06f77bed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:23.284759 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:23.284759 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:23.284759 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:23.284759 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:23.284759 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:23.284759 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:23.284759 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:23.284759 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:23.284759 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:23.284759 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:23.284759 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:23.284759 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:20:23.284759 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:20:23.284759 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:20:23.285666 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:20:23.285666 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:23.285666 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:23.285666 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:23.285666 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:23.285666 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:23.285666 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:23.285666 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:23.285666 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:23.285666 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:23.285666 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:23.285666 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:23.285666 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:23.285666 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:23.285666 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:23.285666 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:23.285666 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:23.286641 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:20:23.286641 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:20:23.286641 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:23.286641 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:23.286641 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 316/611] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:20:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:23.292737 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec3fedb-ea58-4f7f-b374-026f06f77bed None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:23.300578 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 296/612] 10.222.0.85 () {58 vars in 1439 bytes} [Tue Jun 7 14:20:22 2022] PUT /identity/v3/projects/724785eb74e44daf8736fd01fc0d1839/users/7ac4c85918c045b793a1a53300c989a4/roles/17516844e8b947b389d62f50718b852d => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:20:23.430417 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec3fedb-ea58-4f7f-b374-026f06f77bed None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:23.477363 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae87b2f-4bd5-4ca8-be09-5b8bddfc9103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:23.478123 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae87b2f-4bd5-4ca8-be09-5b8bddfc9103 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:23.478409 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae87b2f-4bd5-4ca8-be09-5b8bddfc9103 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:23.478729 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae87b2f-4bd5-4ca8-be09-5b8bddfc9103 None admin] PATH_INFO: `/v3/domains` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:23.479479 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae87b2f-4bd5-4ca8-be09-5b8bddfc9103 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:23.479795 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae87b2f-4bd5-4ca8-be09-5b8bddfc9103 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:23.481287 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae87b2f-4bd5-4ca8-be09-5b8bddfc9103 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:23.484754 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:23.500450 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 317/613] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jun 7 14:20:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:20:23.506142 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404d71ea-79d2-4ca0-a04c-84b085af59e6 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:23.545305 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:23.622104 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404d71ea-79d2-4ca0-a04c-84b085af59e6 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:23.658154 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a123fe-c7d7-441b-a68b-54889243c094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:23.658963 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a123fe-c7d7-441b-a68b-54889243c094 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:23.659251 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a123fe-c7d7-441b-a68b-54889243c094 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:23.659530 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a123fe-c7d7-441b-a68b-54889243c094 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:23.667474 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a123fe-c7d7-441b-a68b-54889243c094 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:23.668976 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a123fe-c7d7-441b-a68b-54889243c094 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:23.685013 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-90a123fe-c7d7-441b-a68b-54889243c094 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 07 14:20:23.685013 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:23.685013 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:23.685013 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:23.685013 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:23.685013 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:23.685013 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:23.685013 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:23.685013 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:23.685013 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:23.685013 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:23.685013 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:23.685013 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 14:20:23.685013 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 14:20:23.685013 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 14:20:23.685013 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 14:20:23.685986 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:23.685986 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:23.685986 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 14:20:23.685986 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:20:23.685986 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 14:20:23.685986 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:20:23.685986 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:23.685986 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:23.685986 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:23.685986 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:23.685986 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:23.685986 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:23.685986 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:23.685986 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:23.685986 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:23.685986 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:23.685986 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:23.685986 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:23.686909 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:23.686909 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:23.686909 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 14:20:23.686909 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 14:20:23.686909 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 14:20:23.686909 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 14:20:23.686909 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 14:20:23.686909 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 14:20:23.686909 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 07 14:20:23.686909 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:23.686909 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 297/614] 10.222.0.85 () {56 vars in 1096 bytes} [Tue Jun 7 14:20:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:23.691869 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae87b2f-4bd5-4ca8-be09-5b8bddfc9103 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:23.840491 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae87b2f-4bd5-4ca8-be09-5b8bddfc9103 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:23.887055 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328f078c-33c8-4da9-a72c-6ded6120c3b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:23.887737 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328f078c-33c8-4da9-a72c-6ded6120c3b6 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:23.888028 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328f078c-33c8-4da9-a72c-6ded6120c3b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:23.888314 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328f078c-33c8-4da9-a72c-6ded6120c3b6 None admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:23.889186 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328f078c-33c8-4da9-a72c-6ded6120c3b6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:23.889502 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328f078c-33c8-4da9-a72c-6ded6120c3b6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:23.891179 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328f078c-33c8-4da9-a72c-6ded6120c3b6 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:23.924078 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 318/615] 10.222.0.85 () {56 vars in 1135 bytes} [Tue Jun 7 14:20:23 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:20:23.929145 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a123fe-c7d7-441b-a68b-54889243c094 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:24.044810 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a123fe-c7d7-441b-a68b-54889243c094 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:24.052831 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:24.082963 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a97b96-4a62-45ad-8ee2-629ad8214aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:24.083798 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a97b96-4a62-45ad-8ee2-629ad8214aa0 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:24.084086 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a97b96-4a62-45ad-8ee2-629ad8214aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:24.084402 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a97b96-4a62-45ad-8ee2-629ad8214aa0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:24.099336 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a97b96-4a62-45ad-8ee2-629ad8214aa0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:24.100955 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a97b96-4a62-45ad-8ee2-629ad8214aa0 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:24.126054 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-e8a97b96-4a62-45ad-8ee2-629ad8214aa0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:24.126054 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:24.126054 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:24.126054 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:24.126054 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:24.126054 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:24.126054 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:24.126054 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:24.126054 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:24.126054 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:24.126054 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:24.126054 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:24.126054 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:20:24.126054 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:20:24.126054 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:20:24.126998 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:20:24.126998 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:24.126998 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:24.126998 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:24.126998 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:24.126998 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:24.126998 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:24.126998 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:24.126998 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:24.126998 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:24.126998 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:24.126998 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:24.126998 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:24.126998 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:24.126998 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:24.126998 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:24.126998 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:24.127852 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:20:24.127852 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:20:24.127852 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:24.127852 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:24.127852 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 298/616] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:20:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:24.133547 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328f078c-33c8-4da9-a72c-6ded6120c3b6 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:24.207014 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:24.269687 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328f078c-33c8-4da9-a72c-6ded6120c3b6 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:24.282506 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a626ff7-4d38-491c-8fc2-1a18be2d6c22 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:24.282816 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a626ff7-4d38-491c-8fc2-1a18be2d6c22 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:24.283073 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a626ff7-4d38-491c-8fc2-1a18be2d6c22 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:24.284745 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 299/617] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:20:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 14:20:24.298484 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2ea219-1817-449e-9dc0-147176758bb6 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:24.298795 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2ea219-1817-449e-9dc0-147176758bb6 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:24.299060 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2ea219-1817-449e-9dc0-147176758bb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:24.311288 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2824fb8c-c548-4249-816d-761f591cd506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:24.312302 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2824fb8c-c548-4249-816d-761f591cd506 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:24.312836 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2824fb8c-c548-4249-816d-761f591cd506 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:24.313268 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2824fb8c-c548-4249-816d-761f591cd506 None admin] PATH_INFO: `/v3/domains` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:24.315046 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2824fb8c-c548-4249-816d-761f591cd506 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:24.315532 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2824fb8c-c548-4249-816d-761f591cd506 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:24.317114 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2824fb8c-c548-4249-816d-761f591cd506 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:24.330373 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 319/618] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jun 7 14:20:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:24.331538 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-0f2ea219-1817-449e-9dc0-147176758bb6 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:20:24.334784 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-0f2ea219-1817-449e-9dc0-147176758bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:20:24.337248 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2824fb8c-c548-4249-816d-761f591cd506 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:24.446542 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 300/619] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:20:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 14:20:24.448245 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beaccbd-b6e5-446c-becc-022ecb2a1e32 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:24.448746 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beaccbd-b6e5-446c-becc-022ecb2a1e32 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:24.449201 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beaccbd-b6e5-446c-becc-022ecb2a1e32 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:24.450966 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 301/620] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:20:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:20:24.465842 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bf18c6-163f-467f-9504-1c25a5858dcb None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:24.466391 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bf18c6-163f-467f-9504-1c25a5858dcb None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:24.466855 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bf18c6-163f-467f-9504-1c25a5858dcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:24.481917 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2824fb8c-c548-4249-816d-761f591cd506 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:24.491821 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-74bf18c6-163f-467f-9504-1c25a5858dcb None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:20:24.493324 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-74bf18c6-163f-467f-9504-1c25a5858dcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:20:24.498311 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:24.525622 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116f6920-9232-46d9-b8e3-17a0112f67ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:24.526655 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116f6920-9232-46d9-b8e3-17a0112f67ca None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:24.526949 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116f6920-9232-46d9-b8e3-17a0112f67ca None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:24.527232 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116f6920-9232-46d9-b8e3-17a0112f67ca None admin] PATH_INFO: `/v3/projects/service` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:24.535664 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116f6920-9232-46d9-b8e3-17a0112f67ca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:24.537624 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116f6920-9232-46d9-b8e3-17a0112f67ca None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:24.554929 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-116f6920-9232-46d9-b8e3-17a0112f67ca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:20:24.554929 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:24.554929 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:24.554929 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:24.554929 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:24.554929 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:24.554929 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:24.554929 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:24.554929 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:24.554929 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:24.554929 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:24.554929 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:24.554929 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 14:20:24.554929 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 14:20:24.554929 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 14:20:24.555874 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 14:20:24.555874 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:24.555874 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:24.555874 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:24.555874 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:24.555874 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:24.555874 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:24.555874 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:24.555874 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:24.555874 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:24.555874 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:24.555874 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:24.555874 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:24.555874 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:24.555874 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:24.555874 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:24.555874 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:24.557166 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 14:20:24.557166 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 14:20:24.557166 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 14:20:24.557166 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 14:20:24.557166 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 14:20:24.557166 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 14:20:24.557166 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:20:24.557166 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:24.557166 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 320/621] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jun 7 14:20:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:24.557724 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116f6920-9232-46d9-b8e3-17a0112f67ca None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:24.572647 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 302/622] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:20:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:20:24.574416 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bf18c6-163f-467f-9504-1c25a5858dcb None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:24.578136 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:24.682471 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116f6920-9232-46d9-b8e3-17a0112f67ca None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:24.690091 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bf18c6-163f-467f-9504-1c25a5858dcb None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:24.727328 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323d9b6d-9273-4669-a74f-475ac75b4833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:24.728136 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323d9b6d-9273-4669-a74f-475ac75b4833 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:24.728474 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323d9b6d-9273-4669-a74f-475ac75b4833 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:24.728748 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323d9b6d-9273-4669-a74f-475ac75b4833 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:24.731006 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4482b60b-02c8-40f4-bfd9-c3e4b13cc968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:24.731998 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4482b60b-02c8-40f4-bfd9-c3e4b13cc968 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:24.732524 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4482b60b-02c8-40f4-bfd9-c3e4b13cc968 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:24.733007 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4482b60b-02c8-40f4-bfd9-c3e4b13cc968 None admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:24.734062 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4482b60b-02c8-40f4-bfd9-c3e4b13cc968 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:24.734590 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4482b60b-02c8-40f4-bfd9-c3e4b13cc968 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:24.736346 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4482b60b-02c8-40f4-bfd9-c3e4b13cc968 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:24.740543 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323d9b6d-9273-4669-a74f-475ac75b4833 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:20:24.750946 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-323d9b6d-9273-4669-a74f-475ac75b4833 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:20:24.750946 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:24.750946 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:24.750946 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:24.750946 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:24.750946 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:24.750946 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:24.750946 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:24.750946 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:24.750946 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:24.750946 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:24.750946 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:24.750946 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:20:24.750946 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:20:24.750946 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:20:24.750946 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:20:24.751897 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:20:24.751897 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:20:24.751897 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:20:24.751897 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:20:24.751897 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:24.751897 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:24.751897 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:24.751897 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:24.751897 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:24.751897 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:24.751897 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:24.751897 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:24.751897 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:24.751897 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:24.751897 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:24.751897 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:24.753114 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:24.753114 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:24.753114 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:24.753114 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:24.753114 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:20:24.753114 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:20:24.753114 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:20:24.753114 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:20:24.753114 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:20:24.753114 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:20:24.753114 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:20:24.753114 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:24.756451 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 321/623] 10.222.0.85 () {56 vars in 1102 bytes} [Tue Jun 7 14:20:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:24.758093 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323d9b6d-9273-4669-a74f-475ac75b4833 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:24.763833 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 303/624] 10.222.0.85 () {56 vars in 1148 bytes} [Tue Jun 7 14:20:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:24.764735 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4482b60b-02c8-40f4-bfd9-c3e4b13cc968 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:24.795394 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:24.847406 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:24.879429 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323d9b6d-9273-4669-a74f-475ac75b4833 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:24.894400 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4482b60b-02c8-40f4-bfd9-c3e4b13cc968 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:24.924282 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882383ac-1ee7-419f-9e6f-67f7c0cb2c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:24.925117 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882383ac-1ee7-419f-9e6f-67f7c0cb2c19 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:24.925414 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882383ac-1ee7-419f-9e6f-67f7c0cb2c19 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:24.925994 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882383ac-1ee7-419f-9e6f-67f7c0cb2c19 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:24.933710 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa3ef8c-71eb-4d08-a241-81f1d1409b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:24.934759 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa3ef8c-71eb-4d08-a241-81f1d1409b90 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:24.935351 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa3ef8c-71eb-4d08-a241-81f1d1409b90 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:24.935831 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa3ef8c-71eb-4d08-a241-81f1d1409b90 None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:24.939442 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882383ac-1ee7-419f-9e6f-67f7c0cb2c19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:20:24.942488 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa3ef8c-71eb-4d08-a241-81f1d1409b90 None admin] RBAC: Adding query filter params (name=service) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:24.943044 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa3ef8c-71eb-4d08-a241-81f1d1409b90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:24.944688 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa3ef8c-71eb-4d08-a241-81f1d1409b90 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:24.946019 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-882383ac-1ee7-419f-9e6f-67f7c0cb2c19 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:20:24.946019 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:24.946019 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:24.946019 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:24.946019 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:24.946019 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:24.946019 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:24.946019 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:24.946019 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:24.946019 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:24.946019 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:24.946019 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:24.946019 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:20:24.946019 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:20:24.946019 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:20:24.946019 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:20:24.946949 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:20:24.946949 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:20:24.946949 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:20:24.946949 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:20:24.946949 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:24.946949 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:24.946949 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:24.946949 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:24.946949 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:24.946949 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:24.946949 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:24.946949 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:24.946949 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:24.946949 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:24.946949 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:24.946949 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:24.947779 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:24.947779 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:24.947779 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:24.947779 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:24.947779 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:20:24.947779 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:20:24.947779 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:20:24.947779 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:20:24.947779 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:20:24.947779 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:20:24.947779 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:20:24.947779 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:24.947779 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 322/625] 10.222.0.85 () {56 vars in 1102 bytes} [Tue Jun 7 14:20:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:24.948494 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882383ac-1ee7-419f-9e6f-67f7c0cb2c19 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:24.963911 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 304/626] 10.222.0.85 () {56 vars in 1103 bytes} [Tue Jun 7 14:20:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:24.965107 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa3ef8c-71eb-4d08-a241-81f1d1409b90 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:25.064294 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:25.075890 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882383ac-1ee7-419f-9e6f-67f7c0cb2c19 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:25.098780 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa3ef8c-71eb-4d08-a241-81f1d1409b90 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:25.119268 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0324cb20-86ab-4b63-b23e-cb60b3337804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:25.120823 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0324cb20-86ab-4b63-b23e-cb60b3337804 None admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:25.121428 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0324cb20-86ab-4b63-b23e-cb60b3337804 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:25.121907 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0324cb20-86ab-4b63-b23e-cb60b3337804 None admin] PATH_INFO: `/v3/projects/724785eb74e44daf8736fd01fc0d1839/users/35aa67025f66459da0c0fc3cbfd45430/roles/17516844e8b947b389d62f50718b852d` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:25.127580 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:25.136815 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2db9371-e705-4217-ad26-35b320432e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:25.137645 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2db9371-e705-4217-ad26-35b320432e68 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:25.137779 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2db9371-e705-4217-ad26-35b320432e68 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:25.138035 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2db9371-e705-4217-ad26-35b320432e68 None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:25.138793 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2db9371-e705-4217-ad26-35b320432e68 None admin] RBAC: Adding query filter params (name=service) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:25.139106 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2db9371-e705-4217-ad26-35b320432e68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:25.140641 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2db9371-e705-4217-ad26-35b320432e68 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:25.154387 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0324cb20-86ab-4b63-b23e-cb60b3337804 None admin] RBAC: Authorizing `identity:create_grant(project_id=724785eb74e44daf8736fd01fc0d1839, user_id=35aa67025f66459da0c0fc3cbfd45430, role_id=17516844e8b947b389d62f50718b852d)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:25.154387 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0324cb20-86ab-4b63-b23e-cb60b3337804 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:25.154601 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 305/627] 10.222.0.85 () {56 vars in 1103 bytes} [Tue Jun 7 14:20:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:25.154601 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2db9371-e705-4217-ad26-35b320432e68 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:25.232976 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 323/628] 10.222.0.85 () {58 vars in 1439 bytes} [Tue Jun 7 14:20:24 2022] PUT /identity/v3/projects/724785eb74e44daf8736fd01fc0d1839/users/35aa67025f66459da0c0fc3cbfd45430/roles/17516844e8b947b389d62f50718b852d => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:20:25.234249 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0324cb20-86ab-4b63-b23e-cb60b3337804 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:25.269471 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2db9371-e705-4217-ad26-35b320432e68 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:25.352734 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5bda2d-4cb0-4808-b155-0f7cc03a164b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:25.353540 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5bda2d-4cb0-4808-b155-0f7cc03a164b None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:25.353832 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5bda2d-4cb0-4808-b155-0f7cc03a164b None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:25.354113 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5bda2d-4cb0-4808-b155-0f7cc03a164b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:25.367954 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5bda2d-4cb0-4808-b155-0f7cc03a164b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:25.370273 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5bda2d-4cb0-4808-b155-0f7cc03a164b None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:25.392342 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-6d5bda2d-4cb0-4808-b155-0f7cc03a164b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:25.392342 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:25.392342 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:25.392342 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:25.392342 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:25.392342 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:25.392342 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:25.392342 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:25.392342 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:25.392342 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:25.392342 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:25.392342 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:25.392342 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:20:25.392342 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:20:25.392342 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:20:25.393400 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:20:25.393400 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:25.393400 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:25.393400 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:25.393400 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:25.393400 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:25.393400 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:25.393400 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:25.393400 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:25.393400 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:25.393400 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:25.393400 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:25.393400 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:25.393400 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:25.393400 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:25.393400 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:25.393400 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:25.394303 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:20:25.394303 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:20:25.394303 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:25.394303 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:25.394303 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 306/629] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:20:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:25.400950 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5bda2d-4cb0-4808-b155-0f7cc03a164b None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:25.471334 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0324cb20-86ab-4b63-b23e-cb60b3337804 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:25.479942 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:25.516260 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c61372-520c-4122-8e66-9972ab988968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:25.518740 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c61372-520c-4122-8e66-9972ab988968 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:25.519231 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c61372-520c-4122-8e66-9972ab988968 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:25.519696 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c61372-520c-4122-8e66-9972ab988968 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:25.533820 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c61372-520c-4122-8e66-9972ab988968 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:25.535780 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c61372-520c-4122-8e66-9972ab988968 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:25.538211 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:25.553033 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5bda2d-4cb0-4808-b155-0f7cc03a164b None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:25.559774 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-b5c61372-520c-4122-8e66-9972ab988968 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:25.559774 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:25.559774 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:25.559774 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:25.559774 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:25.559774 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:25.559774 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:25.559774 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:25.559774 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:25.559774 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:25.559774 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:25.559774 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:25.559774 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:20:25.559774 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:20:25.559774 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:20:25.560699 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:20:25.560699 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:25.560699 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:25.560699 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:25.560699 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:25.560699 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:25.560699 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:25.560699 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:25.560699 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:25.560699 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:25.560699 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:25.560699 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:25.560699 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:25.560699 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:25.560699 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:25.560699 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:25.560699 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:25.561575 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:20:25.561575 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:20:25.561575 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:25.561575 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:25.564354 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 324/630] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:20:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:25.570797 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88d3754-09b0-472a-b420-1e2f4763412c None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:25.571089 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88d3754-09b0-472a-b420-1e2f4763412c None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:25.571372 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88d3754-09b0-472a-b420-1e2f4763412c None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:25.573024 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 325/631] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:20:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:20:25.574091 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88d3754-09b0-472a-b420-1e2f4763412c None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:25.602728 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a2a87e-8e05-4b39-84ab-a1dadd933c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:25.604204 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a2a87e-8e05-4b39-84ab-a1dadd933c7b None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:25.604731 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a2a87e-8e05-4b39-84ab-a1dadd933c7b None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:25.605211 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a2a87e-8e05-4b39-84ab-a1dadd933c7b None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:25.606203 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a2a87e-8e05-4b39-84ab-a1dadd933c7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:25.606735 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a2a87e-8e05-4b39-84ab-a1dadd933c7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:25.608581 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a2a87e-8e05-4b39-84ab-a1dadd933c7b None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:25.623036 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 307/632] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jun 7 14:20:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:25.629986 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088990ed-f7fc-4b3c-9b1e-31fafaa42667 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:25.629986 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088990ed-f7fc-4b3c-9b1e-31fafaa42667 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:25.629986 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088990ed-f7fc-4b3c-9b1e-31fafaa42667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:25.648326 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-088990ed-f7fc-4b3c-9b1e-31fafaa42667 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:20:25.649867 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-088990ed-f7fc-4b3c-9b1e-31fafaa42667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:20:25.701816 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88d3754-09b0-472a-b420-1e2f4763412c None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:25.723689 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 308/633] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:20:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:20:25.724998 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088990ed-f7fc-4b3c-9b1e-31fafaa42667 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:25.755859 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95259bc7-bf4d-4a90-ba2a-a81ef1136f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:25.756677 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95259bc7-bf4d-4a90-ba2a-a81ef1136f82 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:25.757456 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95259bc7-bf4d-4a90-ba2a-a81ef1136f82 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:25.757741 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95259bc7-bf4d-4a90-ba2a-a81ef1136f82 None admin] PATH_INFO: `/v3/domains` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:25.758482 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95259bc7-bf4d-4a90-ba2a-a81ef1136f82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:25.758903 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95259bc7-bf4d-4a90-ba2a-a81ef1136f82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:25.759900 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:25.761481 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95259bc7-bf4d-4a90-ba2a-a81ef1136f82 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:25.781823 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 326/634] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jun 7 14:20:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:25.786957 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95259bc7-bf4d-4a90-ba2a-a81ef1136f82 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:25.844390 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088990ed-f7fc-4b3c-9b1e-31fafaa42667 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:25.845776 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:25.882446 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ce40df-8c7c-473b-8870-858492b8224e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:25.883586 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ce40df-8c7c-473b-8870-858492b8224e None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:25.884091 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ce40df-8c7c-473b-8870-858492b8224e None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:25.884662 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ce40df-8c7c-473b-8870-858492b8224e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:25.892742 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ce40df-8c7c-473b-8870-858492b8224e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:25.894451 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ce40df-8c7c-473b-8870-858492b8224e None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:25.897647 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95259bc7-bf4d-4a90-ba2a-a81ef1136f82 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:25.910641 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-f2ce40df-8c7c-473b-8870-858492b8224e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 07 14:20:25.910641 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:25.910641 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:25.910641 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:25.910641 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:25.910641 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:25.910641 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:25.910641 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:25.910641 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:25.910641 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:25.910641 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:25.910641 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:25.910641 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 14:20:25.910641 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 14:20:25.910641 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 14:20:25.910641 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 14:20:25.911635 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:25.911635 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:25.911635 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 14:20:25.911635 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:20:25.911635 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 14:20:25.911635 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:20:25.911635 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:25.911635 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:25.911635 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:25.911635 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:25.911635 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:25.911635 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:25.911635 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:25.911635 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:25.911635 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:25.911635 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:25.911635 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:25.911635 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:25.912617 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:25.912617 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:25.912617 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 14:20:25.912617 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 14:20:25.912617 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 14:20:25.912617 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 14:20:25.912617 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 14:20:25.912617 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 14:20:25.912617 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 07 14:20:25.912617 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:25.915232 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 309/635] 10.222.0.85 () {56 vars in 1093 bytes} [Tue Jun 7 14:20:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:25.920522 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ce40df-8c7c-473b-8870-858492b8224e None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:25.933914 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af3c932-a0df-4a29-b603-eed0e3e302be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:25.934718 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af3c932-a0df-4a29-b603-eed0e3e302be None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:25.935380 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af3c932-a0df-4a29-b603-eed0e3e302be None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:25.935380 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af3c932-a0df-4a29-b603-eed0e3e302be None admin] PATH_INFO: `/v3/roles/service` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:25.947596 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af3c932-a0df-4a29-b603-eed0e3e302be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:20:25.955437 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-1af3c932-a0df-4a29-b603-eed0e3e302be None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:20:25.955437 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:25.955437 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:25.955437 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:25.955437 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:25.955437 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:25.955437 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:25.955437 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:25.955437 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:25.955437 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:25.955437 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:25.955437 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:25.955437 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:20:25.955437 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:20:25.955437 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:20:25.955437 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:20:25.956662 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:20:25.956662 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:20:25.956662 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:20:25.956662 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:20:25.956662 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:25.956662 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:25.956662 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:25.956662 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:25.956662 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:25.956662 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:25.956662 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:25.956662 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:25.956662 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:25.956662 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:25.956662 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:25.956662 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:25.957530 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:25.957530 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:25.957530 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:25.957530 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:25.957530 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:20:25.957530 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:20:25.957530 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:20:25.957530 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:20:25.957530 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:20:25.957530 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:20:25.957530 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:20:25.957530 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:25.957530 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 327/636] 10.222.0.85 () {56 vars in 1102 bytes} [Tue Jun 7 14:20:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:25.962174 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af3c932-a0df-4a29-b603-eed0e3e302be None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:26.036680 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:26.046611 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ce40df-8c7c-473b-8870-858492b8224e None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:26.083543 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe56b424-f8cf-471a-ac68-0a2299526763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:26.084334 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe56b424-f8cf-471a-ac68-0a2299526763 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:26.084575 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af3c932-a0df-4a29-b603-eed0e3e302be None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:26.084734 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe56b424-f8cf-471a-ac68-0a2299526763 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:26.085145 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe56b424-f8cf-471a-ac68-0a2299526763 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:26.093746 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe56b424-f8cf-471a-ac68-0a2299526763 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:26.095187 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe56b424-f8cf-471a-ac68-0a2299526763 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:26.099957 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:26.111845 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-fe56b424-f8cf-471a-ac68-0a2299526763 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 07 14:20:26.111845 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:26.111845 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:26.111845 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:26.111845 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:26.111845 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:26.111845 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:26.111845 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:26.111845 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:26.111845 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:26.111845 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:26.111845 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:26.111845 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 14:20:26.111845 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 14:20:26.111845 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 14:20:26.111845 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 14:20:26.119431 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:26.119431 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:26.119431 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 14:20:26.119431 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:20:26.119431 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 14:20:26.119431 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:20:26.119431 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:26.119431 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:26.119431 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:26.119431 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:26.119431 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:26.119431 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:26.119431 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:26.119431 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:26.119431 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:26.119431 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:26.119431 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:26.119431 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:26.121122 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:26.121122 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:26.121122 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 14:20:26.121122 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 14:20:26.121122 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 14:20:26.121122 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 14:20:26.121122 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 14:20:26.121122 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 14:20:26.121122 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 07 14:20:26.121122 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:26.121122 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 310/637] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jun 7 14:20:25 2022] GET /identity/v3/users/cinder => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:26.121122 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28d4f8b-ad8b-4e09-80f5-bc0ff06e5987 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:26.121122 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28d4f8b-ad8b-4e09-80f5-bc0ff06e5987 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:26.122189 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28d4f8b-ad8b-4e09-80f5-bc0ff06e5987 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:26.122189 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 311/638] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:20:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:20:26.122189 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28d4f8b-ad8b-4e09-80f5-bc0ff06e5987 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:26.131298 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7782b5f1-3910-4a54-adc7-789df875c988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:26.132005 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7782b5f1-3910-4a54-adc7-789df875c988 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:26.132292 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7782b5f1-3910-4a54-adc7-789df875c988 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:26.132658 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7782b5f1-3910-4a54-adc7-789df875c988 None admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:26.133454 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7782b5f1-3910-4a54-adc7-789df875c988 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:26.133766 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7782b5f1-3910-4a54-adc7-789df875c988 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:26.135217 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7782b5f1-3910-4a54-adc7-789df875c988 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:26.159467 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 328/639] 10.222.0.85 () {56 vars in 1133 bytes} [Tue Jun 7 14:20:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:26.159467 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7782b5f1-3910-4a54-adc7-789df875c988 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:26.237899 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28d4f8b-ad8b-4e09-80f5-bc0ff06e5987 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:26.265530 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7782b5f1-3910-4a54-adc7-789df875c988 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:26.274062 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5324595a-804f-4467-8af7-dc4bb0415474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:26.274725 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5324595a-804f-4467-8af7-dc4bb0415474 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:26.275013 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5324595a-804f-4467-8af7-dc4bb0415474 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:26.275282 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5324595a-804f-4467-8af7-dc4bb0415474 None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:26.276028 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5324595a-804f-4467-8af7-dc4bb0415474 None admin] RBAC: Adding query filter params (name=service) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:26.276333 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5324595a-804f-4467-8af7-dc4bb0415474 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:26.277735 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5324595a-804f-4467-8af7-dc4bb0415474 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:26.286562 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 312/640] 10.222.0.85 () {56 vars in 1103 bytes} [Tue Jun 7 14:20:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:26.289780 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b7a296-9e33-47be-a46f-f8eb67b6fca0 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:26.290299 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b7a296-9e33-47be-a46f-f8eb67b6fca0 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:26.290707 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b7a296-9e33-47be-a46f-f8eb67b6fca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:26.306897 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944dc7f5-b728-45f6-9f96-530e1fd260ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:26.307593 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944dc7f5-b728-45f6-9f96-530e1fd260ef None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:26.307876 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944dc7f5-b728-45f6-9f96-530e1fd260ef None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:26.308151 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944dc7f5-b728-45f6-9f96-530e1fd260ef None admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:26.309019 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944dc7f5-b728-45f6-9f96-530e1fd260ef None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:26.309271 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-79b7a296-9e33-47be-a46f-f8eb67b6fca0 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:20:26.309372 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944dc7f5-b728-45f6-9f96-530e1fd260ef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:26.310443 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-79b7a296-9e33-47be-a46f-f8eb67b6fca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:20:26.310735 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944dc7f5-b728-45f6-9f96-530e1fd260ef None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:26.318198 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:26.332913 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 329/641] 10.222.0.85 () {56 vars in 1137 bytes} [Tue Jun 7 14:20:26 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:26.332913 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944dc7f5-b728-45f6-9f96-530e1fd260ef None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:26.365551 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:26.390563 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 313/642] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:20:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:20:26.391619 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b7a296-9e33-47be-a46f-f8eb67b6fca0 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:26.443293 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944dc7f5-b728-45f6-9f96-530e1fd260ef None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:26.480884 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2fdc0c-3ecc-4987-aea2-e74dcc42cb70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:26.481596 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2fdc0c-3ecc-4987-aea2-e74dcc42cb70 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:26.481884 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2fdc0c-3ecc-4987-aea2-e74dcc42cb70 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:26.482159 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2fdc0c-3ecc-4987-aea2-e74dcc42cb70 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:26.521435 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b7a296-9e33-47be-a46f-f8eb67b6fca0 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:26.559236 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2fdc0c-3ecc-4987-aea2-e74dcc42cb70 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:26.559814 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beeff7b-c1cd-40cc-b4d1-543d4cf12d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:26.560711 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2fdc0c-3ecc-4987-aea2-e74dcc42cb70 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:26.560812 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beeff7b-c1cd-40cc-b4d1-543d4cf12d09 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:26.561208 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beeff7b-c1cd-40cc-b4d1-543d4cf12d09 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:26.561632 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beeff7b-c1cd-40cc-b4d1-543d4cf12d09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:26.572571 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1beeff7b-c1cd-40cc-b4d1-543d4cf12d09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:26.574164 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1beeff7b-c1cd-40cc-b4d1-543d4cf12d09 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:26.576282 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:26.597207 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:26.600262 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-1beeff7b-c1cd-40cc-b4d1-543d4cf12d09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:26.600262 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:26.600262 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:26.600262 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:26.600262 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:26.600262 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:26.600262 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:26.600262 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:26.600262 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:26.600262 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:26.600262 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:26.600262 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:26.600262 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:20:26.600262 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:20:26.600262 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:20:26.601965 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:20:26.601965 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:26.601965 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:26.601965 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:26.601965 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:26.601965 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:26.601965 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:26.601965 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:26.601965 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:26.601965 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:26.601965 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:26.601965 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:26.601965 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:26.601965 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:26.601965 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:26.601965 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:26.601965 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:26.603532 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:20:26.603532 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:20:26.603532 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:26.603532 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:26.603951 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 314/643] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:20:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:26.604908 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beeff7b-c1cd-40cc-b4d1-543d4cf12d09 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:26.689081 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 330/644] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jun 7 14:20:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 14:20:26.690159 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2fdc0c-3ecc-4987-aea2-e74dcc42cb70 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:26.729945 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beeff7b-c1cd-40cc-b4d1-543d4cf12d09 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:26.770868 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42c25c0-b81b-4cea-a56f-7dd2698c44d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:26.771686 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c25c0-b81b-4cea-a56f-7dd2698c44d9 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:26.772095 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c25c0-b81b-4cea-a56f-7dd2698c44d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:26.773119 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c25c0-b81b-4cea-a56f-7dd2698c44d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:26.801721 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2fdc0c-3ecc-4987-aea2-e74dcc42cb70 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:26.831597 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:26.838741 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db27f31-7a49-4c72-bd14-06825796b92a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:26.839470 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:26.840277 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db27f31-7a49-4c72-bd14-06825796b92a None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:26.840588 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db27f31-7a49-4c72-bd14-06825796b92a None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:26.840864 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db27f31-7a49-4c72-bd14-06825796b92a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:26.847126 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42c25c0-b81b-4cea-a56f-7dd2698c44d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:26.848595 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42c25c0-b81b-4cea-a56f-7dd2698c44d9 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:26.852492 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db27f31-7a49-4c72-bd14-06825796b92a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:20:26.860609 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-1db27f31-7a49-4c72-bd14-06825796b92a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:20:26.860609 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:26.860609 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:26.860609 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:26.860609 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:26.860609 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:26.860609 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:26.860609 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:26.860609 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:26.860609 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:26.860609 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:26.860609 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:26.860609 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:20:26.860609 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:20:26.860609 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:20:26.860609 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:20:26.861854 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:20:26.861854 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:20:26.861854 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:20:26.861854 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:20:26.861854 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:26.861854 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:26.861854 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:26.861854 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:26.861854 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:26.861854 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:26.861854 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:26.861854 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:26.861854 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:26.861854 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:26.861854 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:26.861854 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:26.862856 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:26.862856 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:26.862856 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:26.862856 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:26.862856 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:20:26.862856 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:20:26.862856 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:20:26.862856 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:20:26.862856 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:20:26.862856 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:20:26.862856 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:20:26.862856 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:26.862856 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 331/645] 10.222.0.85 () {56 vars in 1102 bytes} [Tue Jun 7 14:20:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:26.862856 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db27f31-7a49-4c72-bd14-06825796b92a None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:26.969877 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 315/646] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jun 7 14:20:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 14:20:26.970544 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42c25c0-b81b-4cea-a56f-7dd2698c44d9 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:26.974279 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db27f31-7a49-4c72-bd14-06825796b92a None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:27.028699 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b6e0c3-4ffb-447e-a89e-4e1b3b891b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:27.029606 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b6e0c3-4ffb-447e-a89e-4e1b3b891b22 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:27.031843 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b6e0c3-4ffb-447e-a89e-4e1b3b891b22 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:27.031843 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b6e0c3-4ffb-447e-a89e-4e1b3b891b22 None admin] PATH_INFO: `/v3/domains` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:27.031843 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b6e0c3-4ffb-447e-a89e-4e1b3b891b22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:27.031843 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b6e0c3-4ffb-447e-a89e-4e1b3b891b22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:27.032897 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b6e0c3-4ffb-447e-a89e-4e1b3b891b22 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:27.045835 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 332/647] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jun 7 14:20:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:27.046487 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b6e0c3-4ffb-447e-a89e-4e1b3b891b22 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:27.094338 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:27.102316 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42c25c0-b81b-4cea-a56f-7dd2698c44d9 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:27.110478 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:27.138684 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6762b20-b834-459f-9ee7-67924d4de20d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:27.139793 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6762b20-b834-459f-9ee7-67924d4de20d None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:27.140475 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6762b20-b834-459f-9ee7-67924d4de20d None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:27.140961 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6762b20-b834-459f-9ee7-67924d4de20d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:27.151309 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6762b20-b834-459f-9ee7-67924d4de20d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:27.152788 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6762b20-b834-459f-9ee7-67924d4de20d None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:27.161517 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b6e0c3-4ffb-447e-a89e-4e1b3b891b22 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:27.171396 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-f6762b20-b834-459f-9ee7-67924d4de20d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:27.171396 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:27.171396 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:27.171396 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:27.171396 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:27.171396 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:27.171396 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:27.171396 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:27.171396 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:27.171396 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:27.171396 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:27.171396 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:27.171396 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:20:27.171396 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:20:27.171396 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:20:27.172410 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:20:27.172410 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:27.172410 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:27.172410 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:27.172410 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:27.172410 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:27.172410 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:27.172410 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:27.172410 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:27.172410 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:27.172410 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:27.172410 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:27.172410 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:27.172410 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:27.172410 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:27.172410 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:27.172410 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:27.178236 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:20:27.178236 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:20:27.178236 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:27.178236 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:27.178236 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 316/648] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:20:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:27.178236 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6762b20-b834-459f-9ee7-67924d4de20d None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:27.204503 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16955f71-0d94-4304-bff8-044e07ce6097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:27.205176 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16955f71-0d94-4304-bff8-044e07ce6097 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:27.205459 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16955f71-0d94-4304-bff8-044e07ce6097 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:27.205736 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16955f71-0d94-4304-bff8-044e07ce6097 None admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:27.206481 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16955f71-0d94-4304-bff8-044e07ce6097 None admin] RBAC: Adding query filter params (name=service) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:27.206787 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16955f71-0d94-4304-bff8-044e07ce6097 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:27.208255 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16955f71-0d94-4304-bff8-044e07ce6097 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:27.217495 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 333/649] 10.222.0.85 () {56 vars in 1103 bytes} [Tue Jun 7 14:20:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:27.218080 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16955f71-0d94-4304-bff8-044e07ce6097 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:27.285474 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6762b20-b834-459f-9ee7-67924d4de20d None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:27.320979 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16955f71-0d94-4304-bff8-044e07ce6097 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:27.323292 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb548b8c-7361-4f08-a37b-71382aaf3f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:27.324089 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb548b8c-7361-4f08-a37b-71382aaf3f6c None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:27.324472 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb548b8c-7361-4f08-a37b-71382aaf3f6c None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:27.324756 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb548b8c-7361-4f08-a37b-71382aaf3f6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:27.334893 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb548b8c-7361-4f08-a37b-71382aaf3f6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:27.336338 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb548b8c-7361-4f08-a37b-71382aaf3f6c None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:27.354085 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-bb548b8c-7361-4f08-a37b-71382aaf3f6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:27.354085 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:27.354085 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:27.354085 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:27.354085 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:27.354085 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:27.354085 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:27.354085 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:27.354085 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:27.354085 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:27.354085 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:27.354085 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:27.354085 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:20:27.354085 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:20:27.354085 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:20:27.355015 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:20:27.355015 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:27.355015 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:27.355015 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:27.355015 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:27.355015 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:27.355015 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:27.355015 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:27.355015 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:27.355015 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:27.355015 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:27.355015 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:27.355015 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:27.355015 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:27.355015 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:27.355015 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:27.355015 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:27.355902 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:20:27.355902 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:20:27.355902 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:27.355902 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:27.355902 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 317/650] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:20:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:27.356183 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb548b8c-7361-4f08-a37b-71382aaf3f6c None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:27.361439 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d41838-1aea-4ba7-9cfc-25b6c95fb83f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:27.362475 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d41838-1aea-4ba7-9cfc-25b6c95fb83f None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:27.362952 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d41838-1aea-4ba7-9cfc-25b6c95fb83f None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:27.363229 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d41838-1aea-4ba7-9cfc-25b6c95fb83f None admin] PATH_INFO: `/v3/users/glance` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:27.372103 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d41838-1aea-4ba7-9cfc-25b6c95fb83f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:27.373637 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d41838-1aea-4ba7-9cfc-25b6c95fb83f None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:27.389617 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-a7d41838-1aea-4ba7-9cfc-25b6c95fb83f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 07 14:20:27.389617 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:27.389617 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:27.389617 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:27.389617 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:27.389617 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:27.389617 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:27.389617 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:27.389617 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:27.389617 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:27.389617 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:27.389617 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:27.389617 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 14:20:27.389617 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 14:20:27.389617 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 14:20:27.389617 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 14:20:27.390701 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:27.390701 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:27.390701 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 14:20:27.390701 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:20:27.390701 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 14:20:27.390701 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:20:27.390701 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:27.390701 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:27.390701 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:27.390701 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:27.390701 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:27.390701 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:27.390701 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:27.390701 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:27.390701 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:27.390701 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:27.390701 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:27.390701 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:27.396422 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:27.396422 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:27.396422 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 14:20:27.396422 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 14:20:27.396422 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 14:20:27.396422 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 14:20:27.396422 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 14:20:27.396422 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 14:20:27.396422 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 07 14:20:27.396422 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:27.396422 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 334/651] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jun 7 14:20:27 2022] GET /identity/v3/users/glance => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:27.396422 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d41838-1aea-4ba7-9cfc-25b6c95fb83f None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:27.456283 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:27.474153 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb548b8c-7361-4f08-a37b-71382aaf3f6c None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:27.495774 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d41838-1aea-4ba7-9cfc-25b6c95fb83f None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:27.511348 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afe372e-14fe-412d-b66d-0925a2f5dd9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:27.512019 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe372e-14fe-412d-b66d-0925a2f5dd9f None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:27.512307 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe372e-14fe-412d-b66d-0925a2f5dd9f None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:27.512609 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe372e-14fe-412d-b66d-0925a2f5dd9f None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:27.513348 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afe372e-14fe-412d-b66d-0925a2f5dd9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:27.513660 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afe372e-14fe-412d-b66d-0925a2f5dd9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:27.515077 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afe372e-14fe-412d-b66d-0925a2f5dd9f None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:27.517455 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:27.527552 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 318/652] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jun 7 14:20:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:27.528747 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afe372e-14fe-412d-b66d-0925a2f5dd9f None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:27.541291 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d82776-ac8b-499c-abe1-ddc1534c3775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:27.542173 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d82776-ac8b-499c-abe1-ddc1534c3775 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:27.543441 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d82776-ac8b-499c-abe1-ddc1534c3775 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:27.543441 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d82776-ac8b-499c-abe1-ddc1534c3775 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:27.555079 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d82776-ac8b-499c-abe1-ddc1534c3775 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:27.556590 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d82776-ac8b-499c-abe1-ddc1534c3775 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:27.577395 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-81d82776-ac8b-499c-abe1-ddc1534c3775 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:27.577395 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:27.577395 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:27.577395 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:27.577395 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:27.577395 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:27.577395 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:27.577395 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:27.577395 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:27.577395 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:27.577395 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:27.577395 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:27.577395 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:20:27.577395 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:20:27.577395 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:20:27.579211 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:20:27.579211 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:27.579211 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:27.579211 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:27.579211 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:27.579211 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:27.579211 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:27.579211 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:27.579211 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:27.579211 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:27.579211 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:27.579211 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:27.579211 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:27.579211 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:27.579211 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:27.579211 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:27.579211 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:27.580508 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:20:27.580508 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:20:27.580508 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:27.580508 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:27.580508 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 335/653] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:20:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:27.580508 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d82776-ac8b-499c-abe1-ddc1534c3775 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:27.599297 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:27.609058 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:27.640908 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afe372e-14fe-412d-b66d-0925a2f5dd9f None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:27.673967 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817fea53-2d9b-498b-b9da-98ad3ed71ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:27.674628 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817fea53-2d9b-498b-b9da-98ad3ed71ab5 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:27.674916 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817fea53-2d9b-498b-b9da-98ad3ed71ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:27.675187 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817fea53-2d9b-498b-b9da-98ad3ed71ab5 None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:27.675912 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817fea53-2d9b-498b-b9da-98ad3ed71ab5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:27.676221 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817fea53-2d9b-498b-b9da-98ad3ed71ab5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:27.677621 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817fea53-2d9b-498b-b9da-98ad3ed71ab5 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:27.684173 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d82776-ac8b-499c-abe1-ddc1534c3775 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:27.689235 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 319/654] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jun 7 14:20:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:27.690100 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817fea53-2d9b-498b-b9da-98ad3ed71ab5 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:27.718359 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdddb419-9bfa-4556-bbb0-08e25de59f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:27.719031 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdddb419-9bfa-4556-bbb0-08e25de59f80 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:27.719319 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdddb419-9bfa-4556-bbb0-08e25de59f80 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:27.719599 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdddb419-9bfa-4556-bbb0-08e25de59f80 None admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:27.720463 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdddb419-9bfa-4556-bbb0-08e25de59f80 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:27.720785 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdddb419-9bfa-4556-bbb0-08e25de59f80 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:27.722202 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdddb419-9bfa-4556-bbb0-08e25de59f80 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:27.739967 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 336/655] 10.222.0.85 () {56 vars in 1137 bytes} [Tue Jun 7 14:20:27 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:27.740739 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdddb419-9bfa-4556-bbb0-08e25de59f80 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:27.789891 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817fea53-2d9b-498b-b9da-98ad3ed71ab5 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:27.807601 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:27.822369 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:27.824962 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252b5255-488b-4b12-8573-44248a20f1e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:27.825759 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252b5255-488b-4b12-8573-44248a20f1e9 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:27.826047 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252b5255-488b-4b12-8573-44248a20f1e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:27.826326 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252b5255-488b-4b12-8573-44248a20f1e9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:27.833089 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252b5255-488b-4b12-8573-44248a20f1e9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:27.834528 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252b5255-488b-4b12-8573-44248a20f1e9 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:27.840363 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdddb419-9bfa-4556-bbb0-08e25de59f80 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:27.849167 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-252b5255-488b-4b12-8573-44248a20f1e9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:20:27.849167 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:27.849167 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:27.849167 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:27.849167 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:27.849167 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:27.849167 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:27.849167 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:27.849167 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:27.849167 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:27.849167 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:27.849167 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:27.849167 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 14:20:27.849167 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 14:20:27.849167 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 14:20:27.850250 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 14:20:27.850250 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:27.850250 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:27.850250 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:27.850250 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:27.850250 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:27.850250 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:27.850250 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:27.850250 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:27.850250 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:27.850250 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:27.850250 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:27.850250 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:27.850250 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:27.850250 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:27.850250 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:27.850250 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:27.851369 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 14:20:27.851369 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 14:20:27.851369 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 14:20:27.851369 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 14:20:27.851369 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 14:20:27.851369 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 14:20:27.851369 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:20:27.851369 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:27.854099 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 320/656] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jun 7 14:20:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:27.861827 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83338f8f-110b-4c99-9f9c-9c70e37e5423 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:27.861827 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83338f8f-110b-4c99-9f9c-9c70e37e5423 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:27.861827 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83338f8f-110b-4c99-9f9c-9c70e37e5423 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:27.861827 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 321/657] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:20:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:20:27.861827 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83338f8f-110b-4c99-9f9c-9c70e37e5423 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:27.876354 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a75e97-0ed3-442c-9fbd-ccb6bb77481d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:27.877049 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a75e97-0ed3-442c-9fbd-ccb6bb77481d None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:27.877339 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a75e97-0ed3-442c-9fbd-ccb6bb77481d None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:27.877617 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a75e97-0ed3-442c-9fbd-ccb6bb77481d None admin] PATH_INFO: `/v3/domains` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:27.878351 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a75e97-0ed3-442c-9fbd-ccb6bb77481d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:27.878678 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a75e97-0ed3-442c-9fbd-ccb6bb77481d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:27.880069 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a75e97-0ed3-442c-9fbd-ccb6bb77481d None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:27.892698 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 337/658] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jun 7 14:20:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:27.893460 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a75e97-0ed3-442c-9fbd-ccb6bb77481d None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:27.967900 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83338f8f-110b-4c99-9f9c-9c70e37e5423 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:27.995539 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a75e97-0ed3-442c-9fbd-ccb6bb77481d None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:28.001574 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5dbeeb-230e-45a7-bbb7-8cf314091762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:28.002356 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5dbeeb-230e-45a7-bbb7-8cf314091762 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:28.002634 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5dbeeb-230e-45a7-bbb7-8cf314091762 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:28.002902 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5dbeeb-230e-45a7-bbb7-8cf314091762 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:28.012984 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5dbeeb-230e-45a7-bbb7-8cf314091762 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:28.015554 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5dbeeb-230e-45a7-bbb7-8cf314091762 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:28.030495 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-ba5dbeeb-230e-45a7-bbb7-8cf314091762 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:20:28.030495 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:28.030495 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:28.030495 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:28.030495 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:28.030495 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:28.030495 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:28.030495 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:28.030495 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:28.030495 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:28.030495 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:28.030495 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:28.030495 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 14:20:28.030495 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 14:20:28.030495 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 14:20:28.031603 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 14:20:28.031603 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:28.031603 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:28.031603 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:28.031603 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:28.031603 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:28.031603 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:28.031603 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:28.031603 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:28.031603 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:28.031603 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:28.031603 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:28.031603 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:28.031603 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:28.031603 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:28.031603 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:28.031603 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:28.032536 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 14:20:28.032536 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 14:20:28.032536 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 14:20:28.032536 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 14:20:28.032536 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 14:20:28.032536 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 14:20:28.032536 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:20:28.032536 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:28.032536 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 322/659] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jun 7 14:20:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:28.032536 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5dbeeb-230e-45a7-bbb7-8cf314091762 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:28.037480 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f0a065-1789-4cf5-a089-5c4e651a7e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:28.038137 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f0a065-1789-4cf5-a089-5c4e651a7e4f None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:28.038425 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f0a065-1789-4cf5-a089-5c4e651a7e4f None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:28.038703 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f0a065-1789-4cf5-a089-5c4e651a7e4f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:28.054764 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:28.074511 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:28.113387 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f0a065-1789-4cf5-a089-5c4e651a7e4f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:28.115053 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f0a065-1789-4cf5-a089-5c4e651a7e4f None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:28.133985 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5dbeeb-230e-45a7-bbb7-8cf314091762 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:28.167570 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225f9dcb-e0c8-4d82-a5f1-f804006a9891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:28.168221 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225f9dcb-e0c8-4d82-a5f1-f804006a9891 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:28.168537 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225f9dcb-e0c8-4d82-a5f1-f804006a9891 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:28.168819 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225f9dcb-e0c8-4d82-a5f1-f804006a9891 None admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:28.169605 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225f9dcb-e0c8-4d82-a5f1-f804006a9891 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:28.169919 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225f9dcb-e0c8-4d82-a5f1-f804006a9891 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:28.171331 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225f9dcb-e0c8-4d82-a5f1-f804006a9891 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:28.183434 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 323/660] 10.222.0.85 () {56 vars in 1148 bytes} [Tue Jun 7 14:20:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:28.184810 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c4cd2-8da1-4e83-beba-06118af14563 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:28.185099 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c4cd2-8da1-4e83-beba-06118af14563 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:28.185382 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c4cd2-8da1-4e83-beba-06118af14563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:28.202668 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-2a6c4cd2-8da1-4e83-beba-06118af14563 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:20:28.203860 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-2a6c4cd2-8da1-4e83-beba-06118af14563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:20:28.216684 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 338/661] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jun 7 14:20:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 14:20:28.217532 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f0a065-1789-4cf5-a089-5c4e651a7e4f None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:28.263875 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:28.272236 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 324/662] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:20:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:20:28.279000 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6c4cd2-8da1-4e83-beba-06118af14563 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:28.307196 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:28.321895 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f0a065-1789-4cf5-a089-5c4e651a7e4f None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:28.356939 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e9abf8-3acc-4489-b05d-a870dd9b1367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:28.357734 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e9abf8-3acc-4489-b05d-a870dd9b1367 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:28.358013 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e9abf8-3acc-4489-b05d-a870dd9b1367 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:28.358288 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e9abf8-3acc-4489-b05d-a870dd9b1367 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:28.365244 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e9abf8-3acc-4489-b05d-a870dd9b1367 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:28.367334 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e9abf8-3acc-4489-b05d-a870dd9b1367 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:28.384032 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-d8e9abf8-3acc-4489-b05d-a870dd9b1367 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 07 14:20:28.384032 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:28.384032 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:28.384032 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:28.384032 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:28.384032 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:28.384032 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:28.384032 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:28.384032 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:28.384032 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:28.384032 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:28.384032 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:28.384032 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 14:20:28.384032 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 14:20:28.384032 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 14:20:28.384032 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 14:20:28.385152 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:28.385152 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:28.385152 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 14:20:28.385152 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:20:28.385152 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 14:20:28.385152 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:20:28.385152 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:28.385152 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:28.385152 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:28.385152 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:28.385152 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:28.385152 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:28.385152 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:28.385152 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:28.385152 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:28.385152 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:28.385152 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:28.385152 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:28.386391 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:28.386391 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:28.386391 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 14:20:28.386391 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 14:20:28.386391 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 14:20:28.386391 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 14:20:28.386391 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 14:20:28.386391 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 14:20:28.386391 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 07 14:20:28.386391 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:28.386391 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 339/663] 10.222.0.85 () {56 vars in 1102 bytes} [Tue Jun 7 14:20:28 2022] GET /identity/v3/users/neutron => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:28.386391 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e9abf8-3acc-4489-b05d-a870dd9b1367 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:28.395704 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6c4cd2-8da1-4e83-beba-06118af14563 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:28.434980 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885ef8c3-db46-48b8-805a-d1c997886913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:28.435648 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885ef8c3-db46-48b8-805a-d1c997886913 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:28.435935 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885ef8c3-db46-48b8-805a-d1c997886913 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:28.436219 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885ef8c3-db46-48b8-805a-d1c997886913 None admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:28.437054 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885ef8c3-db46-48b8-805a-d1c997886913 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:28.437495 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885ef8c3-db46-48b8-805a-d1c997886913 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:28.438784 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885ef8c3-db46-48b8-805a-d1c997886913 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:28.450321 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 325/664] 10.222.0.85 () {56 vars in 1148 bytes} [Tue Jun 7 14:20:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:28.451129 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885ef8c3-db46-48b8-805a-d1c997886913 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:28.488667 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e9abf8-3acc-4489-b05d-a870dd9b1367 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:28.490997 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:28.523425 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadadb01-1310-4045-852f-26ada7304ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:28.523993 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadadb01-1310-4045-852f-26ada7304ec0 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:28.524276 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadadb01-1310-4045-852f-26ada7304ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:28.524652 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadadb01-1310-4045-852f-26ada7304ec0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:28.525409 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadadb01-1310-4045-852f-26ada7304ec0 None admin] RBAC: Adding query filter params (role.id=17516844e8b947b389d62f50718b852d, scope.project.id=724785eb74e44daf8736fd01fc0d1839, user.id=8e857db4902e42989d0d040bbc8517c8) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:28.525720 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadadb01-1310-4045-852f-26ada7304ec0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:28.527579 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadadb01-1310-4045-852f-26ada7304ec0 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:28.534774 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 340/665] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jun 7 14:20:28 2022] GET /identity/v3/role_assignments?user.id=8e857db4902e42989d0d040bbc8517c8&scope.project.id=724785eb74e44daf8736fd01fc0d1839&role.id=17516844e8b947b389d62f50718b852d => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:28.535648 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadadb01-1310-4045-852f-26ada7304ec0 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:28.540917 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:28.556263 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885ef8c3-db46-48b8-805a-d1c997886913 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:28.598527 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab7d6ed-faab-48a0-886b-1864221dd173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:28.599352 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab7d6ed-faab-48a0-886b-1864221dd173 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:28.599648 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab7d6ed-faab-48a0-886b-1864221dd173 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:28.599928 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab7d6ed-faab-48a0-886b-1864221dd173 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:28.611967 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab7d6ed-faab-48a0-886b-1864221dd173 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:28.613497 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab7d6ed-faab-48a0-886b-1864221dd173 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:28.633361 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-0ab7d6ed-faab-48a0-886b-1864221dd173 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:28.633361 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:28.633361 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:28.633361 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:28.633361 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:28.633361 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:28.633361 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:28.633361 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:28.633361 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:28.633361 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:28.633361 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:28.633361 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:28.633361 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:20:28.633361 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:20:28.633361 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:20:28.634344 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:20:28.634344 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:28.634344 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:28.634344 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:28.634344 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:28.634344 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:28.634344 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:28.634344 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:28.634344 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:28.634344 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:28.634344 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:28.634344 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:28.634344 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:28.634344 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:28.634344 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:28.634344 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:28.634344 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:28.635228 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:20:28.635228 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:20:28.635228 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:28.635228 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:28.635228 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 326/666] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:20:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:28.635504 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab7d6ed-faab-48a0-886b-1864221dd173 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:28.669914 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadadb01-1310-4045-852f-26ada7304ec0 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:28.710468 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906f72c7-ca27-4149-bf17-d2b1d82addb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:28.711287 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f72c7-ca27-4149-bf17-d2b1d82addb3 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:28.711746 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f72c7-ca27-4149-bf17-d2b1d82addb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:28.712033 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f72c7-ca27-4149-bf17-d2b1d82addb3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:28.724704 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906f72c7-ca27-4149-bf17-d2b1d82addb3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:20:28.733240 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-906f72c7-ca27-4149-bf17-d2b1d82addb3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:20:28.733240 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:28.733240 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:28.733240 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:28.733240 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:28.733240 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:28.733240 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:28.733240 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:28.733240 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:28.733240 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:28.733240 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:28.733240 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:28.733240 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:20:28.733240 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:20:28.733240 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:20:28.733240 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:20:28.734193 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:20:28.734193 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:20:28.734193 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:20:28.734193 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:20:28.734193 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:28.734193 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:28.734193 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:28.734193 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:28.734193 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:28.734193 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:28.734193 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:28.734193 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:28.734193 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:28.734193 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:28.734193 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:28.734193 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:28.735056 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:28.735056 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:28.735056 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:28.735056 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:28.735056 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:20:28.735056 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:20:28.735056 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:20:28.735056 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:20:28.735056 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:20:28.735056 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:20:28.735056 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:20:28.735056 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:28.735056 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 341/667] 10.222.0.85 () {56 vars in 1102 bytes} [Tue Jun 7 14:20:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:28.735056 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906f72c7-ca27-4149-bf17-d2b1d82addb3 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:28.760764 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab7d6ed-faab-48a0-886b-1864221dd173 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:28.771008 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:28.800518 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd24bf3-e222-41c9-a718-229cddf91d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:28.800518 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd24bf3-e222-41c9-a718-229cddf91d70 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:28.800518 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd24bf3-e222-41c9-a718-229cddf91d70 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:28.800518 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd24bf3-e222-41c9-a718-229cddf91d70 None admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:28.801384 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd24bf3-e222-41c9-a718-229cddf91d70 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:28.801693 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd24bf3-e222-41c9-a718-229cddf91d70 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:28.803112 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd24bf3-e222-41c9-a718-229cddf91d70 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:28.816113 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:28.829206 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 327/668] 10.222.0.85 () {56 vars in 1139 bytes} [Tue Jun 7 14:20:28 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:28.830428 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd24bf3-e222-41c9-a718-229cddf91d70 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:28.859114 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906f72c7-ca27-4149-bf17-d2b1d82addb3 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:28.900875 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f025614-9a33-4956-92ac-71c1f78e0632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:28.901532 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f025614-9a33-4956-92ac-71c1f78e0632 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:28.901815 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f025614-9a33-4956-92ac-71c1f78e0632 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:28.902093 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f025614-9a33-4956-92ac-71c1f78e0632 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:28.902920 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f025614-9a33-4956-92ac-71c1f78e0632 None admin] RBAC: Adding query filter params (role.id=17516844e8b947b389d62f50718b852d, scope.project.id=724785eb74e44daf8736fd01fc0d1839, user.id=89d81f597cb44c56a966537089bcce4e) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:28.903230 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f025614-9a33-4956-92ac-71c1f78e0632 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:28.904679 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f025614-9a33-4956-92ac-71c1f78e0632 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:28.911378 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 342/669] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jun 7 14:20:28 2022] GET /identity/v3/role_assignments?user.id=89d81f597cb44c56a966537089bcce4e&scope.project.id=724785eb74e44daf8736fd01fc0d1839&role.id=17516844e8b947b389d62f50718b852d => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:28.912188 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f025614-9a33-4956-92ac-71c1f78e0632 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:28.975271 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd24bf3-e222-41c9-a718-229cddf91d70 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:29.019485 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a5ced8-3773-44f6-b19c-3380321c9e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:29.020279 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a5ced8-3773-44f6-b19c-3380321c9e75 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:29.020605 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a5ced8-3773-44f6-b19c-3380321c9e75 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:29.020888 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a5ced8-3773-44f6-b19c-3380321c9e75 None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:29.021637 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a5ced8-3773-44f6-b19c-3380321c9e75 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:29.021949 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a5ced8-3773-44f6-b19c-3380321c9e75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:29.024060 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a5ced8-3773-44f6-b19c-3380321c9e75 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:29.034485 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:29.039017 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 328/670] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jun 7 14:20:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:29.040163 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a5ced8-3773-44f6-b19c-3380321c9e75 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:29.052893 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f025614-9a33-4956-92ac-71c1f78e0632 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:29.095096 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1b0244-d8ba-4fdb-b664-c75ae26e5929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:29.095786 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1b0244-d8ba-4fdb-b664-c75ae26e5929 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:29.096073 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1b0244-d8ba-4fdb-b664-c75ae26e5929 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:29.096531 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1b0244-d8ba-4fdb-b664-c75ae26e5929 None admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:29.097264 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1b0244-d8ba-4fdb-b664-c75ae26e5929 None admin] RBAC: Adding query filter params (name=service) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:29.097605 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1b0244-d8ba-4fdb-b664-c75ae26e5929 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:29.099122 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1b0244-d8ba-4fdb-b664-c75ae26e5929 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:29.109381 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 343/671] 10.222.0.85 () {56 vars in 1103 bytes} [Tue Jun 7 14:20:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:29.110522 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1b0244-d8ba-4fdb-b664-c75ae26e5929 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:29.113372 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:29.166925 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a5ced8-3773-44f6-b19c-3380321c9e75 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:29.205734 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d74dca9-d261-4468-9f10-f54a13ba7a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:29.206407 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d74dca9-d261-4468-9f10-f54a13ba7a18 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:29.206697 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d74dca9-d261-4468-9f10-f54a13ba7a18 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:29.206981 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d74dca9-d261-4468-9f10-f54a13ba7a18 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:29.237739 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1b0244-d8ba-4fdb-b664-c75ae26e5929 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:29.283799 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d74dca9-d261-4468-9f10-f54a13ba7a18 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:29.284855 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0734e1-e5b5-4978-8997-270d11417066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:29.285289 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d74dca9-d261-4468-9f10-f54a13ba7a18 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:29.285777 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0734e1-e5b5-4978-8997-270d11417066 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:29.286126 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0734e1-e5b5-4978-8997-270d11417066 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:29.286485 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0734e1-e5b5-4978-8997-270d11417066 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:29.295559 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0734e1-e5b5-4978-8997-270d11417066 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:29.297912 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0734e1-e5b5-4978-8997-270d11417066 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:29.317623 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-6e0734e1-e5b5-4978-8997-270d11417066 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:20:29.317623 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:29.317623 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:29.317623 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:29.317623 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:29.317623 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:29.317623 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:29.317623 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:29.317623 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:29.317623 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:29.317623 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:29.317623 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:29.317623 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 14:20:29.317623 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 14:20:29.317623 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 14:20:29.318523 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 14:20:29.318523 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:29.318523 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:29.318523 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:29.318523 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:29.318523 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:29.318523 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:29.318523 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:29.318523 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:29.318523 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:29.318523 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:29.318523 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:29.318523 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:29.318523 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:29.318523 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:29.318523 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:29.318523 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:29.320772 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 14:20:29.320772 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 14:20:29.320772 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 14:20:29.320772 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 14:20:29.320772 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 14:20:29.320772 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 14:20:29.320772 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:20:29.320772 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:29.320772 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 344/672] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jun 7 14:20:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:29.321904 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0734e1-e5b5-4978-8997-270d11417066 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:29.340622 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:29.347656 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:29.399119 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 329/673] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jun 7 14:20:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 14:20:29.400017 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d74dca9-d261-4468-9f10-f54a13ba7a18 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:29.450006 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0734e1-e5b5-4978-8997-270d11417066 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:29.485794 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb18229b-08eb-4b56-aab6-fa18caac6815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:29.486596 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb18229b-08eb-4b56-aab6-fa18caac6815 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:29.486882 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb18229b-08eb-4b56-aab6-fa18caac6815 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:29.487161 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb18229b-08eb-4b56-aab6-fa18caac6815 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:29.499652 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb18229b-08eb-4b56-aab6-fa18caac6815 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:29.501160 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb18229b-08eb-4b56-aab6-fa18caac6815 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:29.508712 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d74dca9-d261-4468-9f10-f54a13ba7a18 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:29.521132 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-cb18229b-08eb-4b56-aab6-fa18caac6815 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:29.521132 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:29.521132 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:29.521132 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:29.521132 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:29.521132 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:29.521132 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:29.521132 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:29.521132 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:29.521132 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:29.521132 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:29.521132 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:29.521132 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:20:29.521132 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:20:29.521132 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:20:29.523427 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:20:29.523427 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:29.523427 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:29.523427 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:29.523427 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:29.523427 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:29.523427 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:29.523427 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:29.523427 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:29.523427 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:29.523427 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:29.523427 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:29.523427 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:29.523427 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:29.523427 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:29.523427 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:29.523427 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:29.525642 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:20:29.525642 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:20:29.525642 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:29.525642 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:29.525642 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 345/674] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:20:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:29.525642 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb18229b-08eb-4b56-aab6-fa18caac6815 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:29.550587 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f12bc4-71f9-4952-a951-dc30f27a8a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:29.551304 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f12bc4-71f9-4952-a951-dc30f27a8a87 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:29.551588 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f12bc4-71f9-4952-a951-dc30f27a8a87 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:29.551913 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f12bc4-71f9-4952-a951-dc30f27a8a87 None admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:29.552745 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f12bc4-71f9-4952-a951-dc30f27a8a87 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:29.553053 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f12bc4-71f9-4952-a951-dc30f27a8a87 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:29.555031 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f12bc4-71f9-4952-a951-dc30f27a8a87 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:29.567432 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 330/675] 10.222.0.85 () {56 vars in 1148 bytes} [Tue Jun 7 14:20:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:29.568524 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f12bc4-71f9-4952-a951-dc30f27a8a87 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:29.589087 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:29.617408 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:29.629459 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb18229b-08eb-4b56-aab6-fa18caac6815 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:29.665920 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b653d2a-6f47-45ec-b328-a0e7db9c5343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:29.666977 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b653d2a-6f47-45ec-b328-a0e7db9c5343 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:29.667681 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b653d2a-6f47-45ec-b328-a0e7db9c5343 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:29.667966 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b653d2a-6f47-45ec-b328-a0e7db9c5343 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:29.673898 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f12bc4-71f9-4952-a951-dc30f27a8a87 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:29.681172 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b653d2a-6f47-45ec-b328-a0e7db9c5343 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:29.683067 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b653d2a-6f47-45ec-b328-a0e7db9c5343 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:29.708390 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-9b653d2a-6f47-45ec-b328-a0e7db9c5343 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:29.708390 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:29.708390 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:29.708390 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:29.708390 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:29.708390 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:29.708390 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:29.708390 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:29.708390 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:29.708390 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:29.708390 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:29.708390 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:29.708390 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:20:29.708390 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:20:29.708390 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:20:29.710550 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:20:29.710550 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:29.710550 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:29.710550 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:29.710550 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:29.710550 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:29.710550 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:29.710550 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:29.710550 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:29.710550 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:29.710550 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:29.710550 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:29.710550 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:29.710550 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:29.710550 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:29.710550 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:29.710550 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:29.711775 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:20:29.711775 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:20:29.711775 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:29.711775 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:29.713999 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 346/676] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:20:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:29.713999 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b653d2a-6f47-45ec-b328-a0e7db9c5343 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:29.716825 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be072199-e86d-4494-9e0c-50d74af88337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:29.716825 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be072199-e86d-4494-9e0c-50d74af88337 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:29.716825 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be072199-e86d-4494-9e0c-50d74af88337 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:29.716825 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be072199-e86d-4494-9e0c-50d74af88337 None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:29.717789 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be072199-e86d-4494-9e0c-50d74af88337 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:29.718193 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be072199-e86d-4494-9e0c-50d74af88337 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:29.719986 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be072199-e86d-4494-9e0c-50d74af88337 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:29.738923 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 331/677] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jun 7 14:20:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:29.740193 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be072199-e86d-4494-9e0c-50d74af88337 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:29.834328 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:29.857442 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b653d2a-6f47-45ec-b328-a0e7db9c5343 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:29.857952 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be072199-e86d-4494-9e0c-50d74af88337 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:29.877567 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:29.895290 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df179d6e-f7af-4f09-aa20-6f35b7d034c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:29.896130 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df179d6e-f7af-4f09-aa20-6f35b7d034c5 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:29.896519 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df179d6e-f7af-4f09-aa20-6f35b7d034c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:29.896756 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f22d0e-03fe-441b-9d5e-4d16959dffda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:29.896983 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df179d6e-f7af-4f09-aa20-6f35b7d034c5 None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:29.897417 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f22d0e-03fe-441b-9d5e-4d16959dffda None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:29.897708 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f22d0e-03fe-441b-9d5e-4d16959dffda None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:29.897990 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f22d0e-03fe-441b-9d5e-4d16959dffda None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:29.898122 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df179d6e-f7af-4f09-aa20-6f35b7d034c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:29.898539 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df179d6e-f7af-4f09-aa20-6f35b7d034c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:29.898823 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f22d0e-03fe-441b-9d5e-4d16959dffda None admin] RBAC: Adding query filter params (role.id=17516844e8b947b389d62f50718b852d, scope.project.id=724785eb74e44daf8736fd01fc0d1839, user.id=b51da9867d9147f198b8077e953c4f9a) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:29.899141 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f22d0e-03fe-441b-9d5e-4d16959dffda None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:29.900207 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df179d6e-f7af-4f09-aa20-6f35b7d034c5 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:29.900581 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f22d0e-03fe-441b-9d5e-4d16959dffda None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:29.907156 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 347/678] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jun 7 14:20:29 2022] GET /identity/v3/role_assignments?user.id=b51da9867d9147f198b8077e953c4f9a&scope.project.id=724785eb74e44daf8736fd01fc0d1839&role.id=17516844e8b947b389d62f50718b852d => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:29.908181 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f22d0e-03fe-441b-9d5e-4d16959dffda None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:29.915659 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 332/679] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jun 7 14:20:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:29.921408 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df179d6e-f7af-4f09-aa20-6f35b7d034c5 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:30.054940 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df179d6e-f7af-4f09-aa20-6f35b7d034c5 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:30.096333 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8d813d-2fa1-4695-8560-7734e3a7d4a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:30.097092 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8d813d-2fa1-4695-8560-7734e3a7d4a5 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:30.097377 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8d813d-2fa1-4695-8560-7734e3a7d4a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:30.097655 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8d813d-2fa1-4695-8560-7734e3a7d4a5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:30.107475 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8d813d-2fa1-4695-8560-7734e3a7d4a5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:30.109018 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8d813d-2fa1-4695-8560-7734e3a7d4a5 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:30.112148 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:30.127806 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-de8d813d-2fa1-4695-8560-7734e3a7d4a5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:20:30.127806 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:30.127806 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:30.127806 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:30.127806 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:30.127806 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:30.127806 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:30.127806 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:30.127806 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:30.127806 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:30.127806 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:30.127806 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:30.127806 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 14:20:30.127806 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 14:20:30.127806 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 14:20:30.136928 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 14:20:30.136928 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:30.136928 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:30.136928 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:30.136928 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:30.136928 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:30.136928 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:30.136928 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:30.136928 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:30.136928 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:30.136928 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:30.136928 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:30.136928 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:30.136928 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:30.136928 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:30.136928 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:30.136928 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:30.139162 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 14:20:30.139162 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 14:20:30.139162 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 14:20:30.139162 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 14:20:30.139162 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 14:20:30.139162 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 14:20:30.139162 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:20:30.139162 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:30.139162 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 333/680] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jun 7 14:20:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:20:30.140301 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f22d0e-03fe-441b-9d5e-4d16959dffda None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:30.140501 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8d813d-2fa1-4695-8560-7734e3a7d4a5 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:30.209026 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33b3d9d-1ffa-485c-a0d1-1f5ef6c97ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:30.210412 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33b3d9d-1ffa-485c-a0d1-1f5ef6c97ca1 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:30.210703 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33b3d9d-1ffa-485c-a0d1-1f5ef6c97ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:30.210985 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33b3d9d-1ffa-485c-a0d1-1f5ef6c97ca1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:30.219532 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33b3d9d-1ffa-485c-a0d1-1f5ef6c97ca1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:30.221203 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33b3d9d-1ffa-485c-a0d1-1f5ef6c97ca1 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:30.238383 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-d33b3d9d-1ffa-485c-a0d1-1f5ef6c97ca1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 07 14:20:30.238383 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:30.238383 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:30.238383 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:30.238383 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:30.238383 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:30.238383 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:30.238383 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:30.238383 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:30.238383 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:30.238383 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:30.238383 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:30.238383 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 14:20:30.238383 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 14:20:30.238383 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 14:20:30.238383 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 14:20:30.240363 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:30.240363 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:30.240363 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 14:20:30.240363 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:20:30.240363 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 14:20:30.240363 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:20:30.240363 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:30.240363 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:30.240363 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:30.240363 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:30.240363 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:30.240363 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:30.240363 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:30.240363 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:30.240363 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:30.240363 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:30.240363 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:30.240363 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:30.241954 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:30.241954 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:30.241954 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 14:20:30.241954 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 14:20:30.241954 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 14:20:30.241954 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 14:20:30.241954 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 14:20:30.241954 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 14:20:30.241954 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 07 14:20:30.241954 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:30.248288 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 348/681] 10.222.0.85 () {56 vars in 1096 bytes} [Tue Jun 7 14:20:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:30.249773 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33b3d9d-1ffa-485c-a0d1-1f5ef6c97ca1 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:30.304769 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:30.306486 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8d813d-2fa1-4695-8560-7734e3a7d4a5 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:30.353372 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fec2630-16b2-42f8-b306-1304c03bd47b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:30.354388 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fec2630-16b2-42f8-b306-1304c03bd47b None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:30.354877 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fec2630-16b2-42f8-b306-1304c03bd47b None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:30.355355 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fec2630-16b2-42f8-b306-1304c03bd47b None admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:30.356777 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fec2630-16b2-42f8-b306-1304c03bd47b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:30.357637 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fec2630-16b2-42f8-b306-1304c03bd47b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:30.360932 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fec2630-16b2-42f8-b306-1304c03bd47b None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:30.378873 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 334/682] 10.222.0.85 () {56 vars in 1148 bytes} [Tue Jun 7 14:20:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:30.382845 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33b3d9d-1ffa-485c-a0d1-1f5ef6c97ca1 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:30.384798 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fec2630-16b2-42f8-b306-1304c03bd47b None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:30.423081 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491444ea-9fef-4cea-adcb-ad2c26a120dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:30.423804 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491444ea-9fef-4cea-adcb-ad2c26a120dc None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:30.424095 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491444ea-9fef-4cea-adcb-ad2c26a120dc None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:30.424514 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491444ea-9fef-4cea-adcb-ad2c26a120dc None admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:30.425415 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491444ea-9fef-4cea-adcb-ad2c26a120dc None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:30.425743 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491444ea-9fef-4cea-adcb-ad2c26a120dc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:30.427233 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491444ea-9fef-4cea-adcb-ad2c26a120dc None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:30.452142 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 349/683] 10.222.0.85 () {56 vars in 1135 bytes} [Tue Jun 7 14:20:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:20:30.454145 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491444ea-9fef-4cea-adcb-ad2c26a120dc None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:30.471985 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:30.526829 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fec2630-16b2-42f8-b306-1304c03bd47b None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:30.567741 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0c159d-a348-4cea-9577-167a8f7a6a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:30.568489 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0c159d-a348-4cea-9577-167a8f7a6a61 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:30.568850 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0c159d-a348-4cea-9577-167a8f7a6a61 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:30.570047 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0c159d-a348-4cea-9577-167a8f7a6a61 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:30.570967 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0c159d-a348-4cea-9577-167a8f7a6a61 None admin] RBAC: Adding query filter params (role.id=17516844e8b947b389d62f50718b852d, scope.project.id=724785eb74e44daf8736fd01fc0d1839, user.id=7ac4c85918c045b793a1a53300c989a4) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:30.571358 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0c159d-a348-4cea-9577-167a8f7a6a61 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:30.573092 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0c159d-a348-4cea-9577-167a8f7a6a61 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:30.580780 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 335/684] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jun 7 14:20:30 2022] GET /identity/v3/role_assignments?user.id=7ac4c85918c045b793a1a53300c989a4&scope.project.id=724785eb74e44daf8736fd01fc0d1839&role.id=17516844e8b947b389d62f50718b852d => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:20:30.586782 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:30.591207 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491444ea-9fef-4cea-adcb-ad2c26a120dc None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:30.632317 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e6acf9-884c-4fa7-9f7c-d38c6afeb89f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:30.633021 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e6acf9-884c-4fa7-9f7c-d38c6afeb89f None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:30.633316 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e6acf9-884c-4fa7-9f7c-d38c6afeb89f None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:30.633605 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e6acf9-884c-4fa7-9f7c-d38c6afeb89f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:30.712318 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e6acf9-884c-4fa7-9f7c-d38c6afeb89f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:30.713842 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e6acf9-884c-4fa7-9f7c-d38c6afeb89f None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:30.825453 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:30.830350 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 350/685] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jun 7 14:20:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 14:20:30.837595 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0c159d-a348-4cea-9577-167a8f7a6a61 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:30.985651 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0c159d-a348-4cea-9577-167a8f7a6a61 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:31.012428 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:31.025607 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c254b9d-14a1-47ec-a95f-d78f34ee6375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:31.026757 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c254b9d-14a1-47ec-a95f-d78f34ee6375 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:31.027204 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c254b9d-14a1-47ec-a95f-d78f34ee6375 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:31.027641 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c254b9d-14a1-47ec-a95f-d78f34ee6375 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:31.039412 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c254b9d-14a1-47ec-a95f-d78f34ee6375 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:31.041228 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c254b9d-14a1-47ec-a95f-d78f34ee6375 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:31.069757 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-1c254b9d-14a1-47ec-a95f-d78f34ee6375 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:31.069757 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:31.069757 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:31.069757 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:31.069757 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:31.069757 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:31.069757 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:31.069757 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:31.069757 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:31.069757 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:31.069757 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:31.069757 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:31.069757 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:20:31.069757 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:20:31.069757 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:20:31.070809 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:20:31.070809 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:31.070809 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:31.070809 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:31.070809 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:31.070809 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:31.070809 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:31.070809 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:31.070809 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:31.070809 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:31.070809 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:31.070809 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:31.070809 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:31.070809 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:31.070809 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:31.070809 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:31.070809 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:31.071702 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:20:31.071702 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:20:31.071702 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:31.071702 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:31.073385 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 336/686] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:20:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:31.079210 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e6acf9-884c-4fa7-9f7c-d38c6afeb89f None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:31.194768 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8aa120-23cf-4a8b-8ba0-5a9c8bc613c9 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:31.195442 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8aa120-23cf-4a8b-8ba0-5a9c8bc613c9 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:31.195739 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8aa120-23cf-4a8b-8ba0-5a9c8bc613c9 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:31.197308 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 337/687] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:20:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 14:20:31.200115 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e6acf9-884c-4fa7-9f7c-d38c6afeb89f None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:31.203889 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fcfcd4-db64-44eb-acf5-ecc5af42eb77 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:31.204240 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fcfcd4-db64-44eb-acf5-ecc5af42eb77 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:31.204652 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fcfcd4-db64-44eb-acf5-ecc5af42eb77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:31.237562 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eb6d25-3d91-4bb5-a9ca-b6858aad63b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:31.238481 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eb6d25-3d91-4bb5-a9ca-b6858aad63b1 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:31.238912 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eb6d25-3d91-4bb5-a9ca-b6858aad63b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:31.239328 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eb6d25-3d91-4bb5-a9ca-b6858aad63b1 None admin] PATH_INFO: `/v3/domains` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:31.240195 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eb6d25-3d91-4bb5-a9ca-b6858aad63b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:31.240754 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eb6d25-3d91-4bb5-a9ca-b6858aad63b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:31.242439 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eb6d25-3d91-4bb5-a9ca-b6858aad63b1 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:31.245149 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-18fcfcd4-db64-44eb-acf5-ecc5af42eb77 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:20:31.252786 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-18fcfcd4-db64-44eb-acf5-ecc5af42eb77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:20:31.255713 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 351/688] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jun 7 14:20:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:31.262335 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eb6d25-3d91-4bb5-a9ca-b6858aad63b1 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:31.380896 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:31.382363 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 338/689] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:20:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:20:31.408008 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fcfcd4-db64-44eb-acf5-ecc5af42eb77 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:31.453529 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eb6d25-3d91-4bb5-a9ca-b6858aad63b1 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:31.494796 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f78c75f-5c38-44d3-b077-a6b7bccedeb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:31.495523 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f78c75f-5c38-44d3-b077-a6b7bccedeb2 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:31.495810 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f78c75f-5c38-44d3-b077-a6b7bccedeb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:31.496087 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f78c75f-5c38-44d3-b077-a6b7bccedeb2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:31.503804 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f78c75f-5c38-44d3-b077-a6b7bccedeb2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:31.505470 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f78c75f-5c38-44d3-b077-a6b7bccedeb2 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:31.524989 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-1f78c75f-5c38-44d3-b077-a6b7bccedeb2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:20:31.524989 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:31.524989 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:31.524989 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:31.524989 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:31.524989 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:31.524989 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:31.524989 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:31.524989 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:31.524989 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:31.524989 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:31.524989 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:31.524989 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 14:20:31.524989 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 14:20:31.524989 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 14:20:31.525940 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 14:20:31.525940 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:31.525940 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:31.525940 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:31.525940 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:31.525940 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:31.525940 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:31.525940 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:31.525940 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:31.525940 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:31.525940 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:31.525940 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:31.525940 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:31.525940 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:31.525940 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:31.525940 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:31.525940 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:31.526810 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 14:20:31.526810 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 14:20:31.526810 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 14:20:31.526810 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 14:20:31.526810 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 14:20:31.526810 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 14:20:31.526810 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:20:31.526810 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:31.526810 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 352/690] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jun 7 14:20:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:31.532396 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92189dfa-9896-4936-966d-515065ce6833 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:31.532396 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92189dfa-9896-4936-966d-515065ce6833 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:31.532396 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92189dfa-9896-4936-966d-515065ce6833 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:31.532396 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 353/691] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:20:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:20:31.536185 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92189dfa-9896-4936-966d-515065ce6833 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:31.599100 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fcfcd4-db64-44eb-acf5-ecc5af42eb77 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:31.602710 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:31.644632 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b30cea-35e2-4f21-851f-b1f26ef80922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:31.645697 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b30cea-35e2-4f21-851f-b1f26ef80922 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:31.645992 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b30cea-35e2-4f21-851f-b1f26ef80922 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:31.646277 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b30cea-35e2-4f21-851f-b1f26ef80922 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:31.651518 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92189dfa-9896-4936-966d-515065ce6833 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:31.659634 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b30cea-35e2-4f21-851f-b1f26ef80922 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:20:31.671717 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-f8b30cea-35e2-4f21-851f-b1f26ef80922 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:20:31.671717 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:31.671717 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:31.671717 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:31.671717 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:31.671717 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:31.671717 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:31.671717 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:31.671717 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:31.671717 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:31.671717 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:31.671717 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:31.671717 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:20:31.671717 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:20:31.671717 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:20:31.671717 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:20:31.672951 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:20:31.672951 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:20:31.672951 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:20:31.672951 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:20:31.672951 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:31.672951 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:31.672951 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:31.672951 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:31.672951 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:31.672951 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:31.672951 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:31.672951 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:31.672951 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:31.672951 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:31.672951 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:31.672951 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:31.673802 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:31.673802 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:31.673802 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:31.673802 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:31.673802 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:20:31.673802 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:20:31.673802 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:20:31.673802 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:20:31.673802 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:20:31.673802 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:20:31.673802 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:20:31.673802 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:31.673802 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 339/692] 10.222.0.85 () {56 vars in 1096 bytes} [Tue Jun 7 14:20:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:20:31.677376 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcd0494-731a-4e78-821b-a6c5eca2d786 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:31.677801 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcd0494-731a-4e78-821b-a6c5eca2d786 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:31.678085 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcd0494-731a-4e78-821b-a6c5eca2d786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:31.687334 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:31.708362 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14da6e1d-9183-48c6-b651-f306f450e5ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:31.708362 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14da6e1d-9183-48c6-b651-f306f450e5ca None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:31.708700 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14da6e1d-9183-48c6-b651-f306f450e5ca None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:31.709039 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14da6e1d-9183-48c6-b651-f306f450e5ca None admin] PATH_INFO: `/v3/projects` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:31.710390 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14da6e1d-9183-48c6-b651-f306f450e5ca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:31.710711 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14da6e1d-9183-48c6-b651-f306f450e5ca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:31.711857 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-0dcd0494-731a-4e78-821b-a6c5eca2d786 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:20:31.712227 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14da6e1d-9183-48c6-b651-f306f450e5ca None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:31.713527 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-0dcd0494-731a-4e78-821b-a6c5eca2d786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:20:31.731564 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 354/693] 10.222.0.85 () {56 vars in 1148 bytes} [Tue Jun 7 14:20:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:31.732340 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14da6e1d-9183-48c6-b651-f306f450e5ca None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:31.824480 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 340/694] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:20:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:20:31.824480 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcd0494-731a-4e78-821b-a6c5eca2d786 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:31.862855 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14da6e1d-9183-48c6-b651-f306f450e5ca None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:31.900837 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46df8f56-bd4a-4d0e-a93b-bd660c04743e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:31.901615 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46df8f56-bd4a-4d0e-a93b-bd660c04743e None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:31.901903 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46df8f56-bd4a-4d0e-a93b-bd660c04743e None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:31.902206 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46df8f56-bd4a-4d0e-a93b-bd660c04743e None admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:31.903038 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46df8f56-bd4a-4d0e-a93b-bd660c04743e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:31.903343 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46df8f56-bd4a-4d0e-a93b-bd660c04743e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:31.904758 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46df8f56-bd4a-4d0e-a93b-bd660c04743e None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:31.917442 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 355/695] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jun 7 14:20:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:31.918141 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46df8f56-bd4a-4d0e-a93b-bd660c04743e None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:31.936562 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:31.944933 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcd0494-731a-4e78-821b-a6c5eca2d786 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:31.977471 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:31.985718 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2495daf9-0e18-4094-9af1-3d7169a12c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:31.986593 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2495daf9-0e18-4094-9af1-3d7169a12c53 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:31.987090 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2495daf9-0e18-4094-9af1-3d7169a12c53 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:31.987511 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2495daf9-0e18-4094-9af1-3d7169a12c53 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:31.990452 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2495daf9-0e18-4094-9af1-3d7169a12c53 None admin] RBAC: Adding query filter params (role.id=17516844e8b947b389d62f50718b852d, scope.project.id=724785eb74e44daf8736fd01fc0d1839, user.id=35aa67025f66459da0c0fc3cbfd45430) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:31.990452 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2495daf9-0e18-4094-9af1-3d7169a12c53 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:31.990452 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2495daf9-0e18-4094-9af1-3d7169a12c53 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:31.998381 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 341/696] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jun 7 14:20:31 2022] GET /identity/v3/role_assignments?user.id=35aa67025f66459da0c0fc3cbfd45430&scope.project.id=724785eb74e44daf8736fd01fc0d1839&role.id=17516844e8b947b389d62f50718b852d => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:31.999496 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2495daf9-0e18-4094-9af1-3d7169a12c53 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:32.104591 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46df8f56-bd4a-4d0e-a93b-bd660c04743e None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:32.123885 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2495daf9-0e18-4094-9af1-3d7169a12c53 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:32.161620 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707565c0-b545-484e-9c63-87b770a60a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:32.163406 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707565c0-b545-484e-9c63-87b770a60a07 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:32.163927 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707565c0-b545-484e-9c63-87b770a60a07 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:32.164469 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707565c0-b545-484e-9c63-87b770a60a07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:32.176657 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707565c0-b545-484e-9c63-87b770a60a07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:32.179078 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707565c0-b545-484e-9c63-87b770a60a07 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:32.193190 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cf5abf-58f3-4370-b6ec-2e7a0a06ba25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:32.206210 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cf5abf-58f3-4370-b6ec-2e7a0a06ba25 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:32.206210 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cf5abf-58f3-4370-b6ec-2e7a0a06ba25 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:32.206210 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cf5abf-58f3-4370-b6ec-2e7a0a06ba25 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:32.206210 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cf5abf-58f3-4370-b6ec-2e7a0a06ba25 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:32.206210 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cf5abf-58f3-4370-b6ec-2e7a0a06ba25 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:32.206604 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-707565c0-b545-484e-9c63-87b770a60a07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:32.206604 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:32.206604 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:32.206604 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:32.206604 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:32.206604 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:32.206604 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:32.206604 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:32.206604 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:32.206604 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:32.206604 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:32.206604 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:32.206604 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:20:32.206604 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:20:32.206604 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:20:32.207440 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:20:32.207440 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:32.207440 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:32.207440 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:32.207440 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:32.207440 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:32.207440 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:32.207440 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:32.207440 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:32.207440 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:32.207440 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:32.207440 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:32.207440 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:32.207440 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:32.207440 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:32.207440 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:32.207440 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:32.208319 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:20:32.208319 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:20:32.208319 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:32.208319 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:32.208319 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 342/697] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:20:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:32.212282 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707565c0-b545-484e-9c63-87b770a60a07 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:32.228543 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-59cf5abf-58f3-4370-b6ec-2e7a0a06ba25 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 07 14:20:32.228543 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:32.228543 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 14:20:32.228543 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 14:20:32.228543 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 14:20:32.228543 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 14:20:32.228543 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 14:20:32.228543 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:20:32.228543 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 07 14:20:32.228543 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:32.228543 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 14:20:32.228543 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:32.228543 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:32.228543 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:32.228543 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:32.228543 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:32.229613 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:32.229613 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:32.229613 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:32.229613 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:32.229613 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:32.229613 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:32.229613 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:32.229613 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 14:20:32.229613 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 14:20:32.229613 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 14:20:32.229613 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 14:20:32.229613 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:32.229613 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:32.229613 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:32.229613 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:32.229613 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:32.231041 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:32.231041 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:32.231041 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:32.231041 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:32.231041 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:32.231041 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:32.231041 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:32.231041 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:32.231041 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:32.231041 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:32.231041 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:32.231041 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 14:20:32.231041 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:20:32.231041 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 07 14:20:32.231041 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:32.234739 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 356/698] 10.222.0.85 () {56 vars in 1129 bytes} [Tue Jun 7 14:20:31 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:32.236019 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cf5abf-58f3-4370-b6ec-2e7a0a06ba25 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:32.252786 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:32.330798 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707565c0-b545-484e-9c63-87b770a60a07 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:32.378216 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5557d28-c5fe-4187-b074-82cbba7d3f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:32.379254 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5557d28-c5fe-4187-b074-82cbba7d3f20 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:32.379742 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5557d28-c5fe-4187-b074-82cbba7d3f20 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:32.380280 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5557d28-c5fe-4187-b074-82cbba7d3f20 None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:32.381321 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5557d28-c5fe-4187-b074-82cbba7d3f20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:32.382393 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5557d28-c5fe-4187-b074-82cbba7d3f20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:32.383986 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5557d28-c5fe-4187-b074-82cbba7d3f20 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:32.397798 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 343/699] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jun 7 14:20:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:20:32.403472 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5557d28-c5fe-4187-b074-82cbba7d3f20 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:32.421644 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cf5abf-58f3-4370-b6ec-2e7a0a06ba25 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:32.432787 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:32.462630 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27e314a-e1da-454c-b7c7-eba5f43dc1c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:32.463303 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27e314a-e1da-454c-b7c7-eba5f43dc1c2 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:32.463591 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27e314a-e1da-454c-b7c7-eba5f43dc1c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:32.463871 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27e314a-e1da-454c-b7c7-eba5f43dc1c2 None admin] PATH_INFO: `/v3/services` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:32.464696 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27e314a-e1da-454c-b7c7-eba5f43dc1c2 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:32.464927 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27e314a-e1da-454c-b7c7-eba5f43dc1c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:32.466316 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27e314a-e1da-454c-b7c7-eba5f43dc1c2 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:32.472983 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 357/700] 10.222.0.85 () {56 vars in 1124 bytes} [Tue Jun 7 14:20:32 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:32.477487 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27e314a-e1da-454c-b7c7-eba5f43dc1c2 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:32.484797 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:32.518109 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5557d28-c5fe-4187-b074-82cbba7d3f20 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:32.555699 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b0339a-fb86-4de2-8e7c-8d08892b4173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:32.556594 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b0339a-fb86-4de2-8e7c-8d08892b4173 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:32.556984 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b0339a-fb86-4de2-8e7c-8d08892b4173 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:32.557268 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b0339a-fb86-4de2-8e7c-8d08892b4173 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:32.566810 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b0339a-fb86-4de2-8e7c-8d08892b4173 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:32.568986 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b0339a-fb86-4de2-8e7c-8d08892b4173 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:32.592207 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-29b0339a-fb86-4de2-8e7c-8d08892b4173 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 07 14:20:32.592207 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:32.592207 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:32.592207 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:32.592207 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:32.592207 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:32.592207 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:32.592207 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:32.592207 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:32.592207 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:32.592207 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:32.592207 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:32.592207 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 14:20:32.592207 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 14:20:32.592207 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 14:20:32.592207 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 14:20:32.593241 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:32.593241 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:32.593241 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 14:20:32.593241 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:20:32.593241 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 14:20:32.593241 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:20:32.593241 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:32.593241 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:32.593241 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:32.593241 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:32.593241 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:32.593241 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:32.593241 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:32.593241 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:32.593241 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:32.593241 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:32.593241 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:32.593241 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:32.594196 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:32.594196 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:32.594196 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 14:20:32.594196 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 14:20:32.594196 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 14:20:32.594196 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 14:20:32.594196 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 14:20:32.594196 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 14:20:32.594196 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 07 14:20:32.594196 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:32.594727 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27e314a-e1da-454c-b7c7-eba5f43dc1c2 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:32.594809 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 344/701] 10.222.0.85 () {56 vars in 1093 bytes} [Tue Jun 7 14:20:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:20:32.599269 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b0339a-fb86-4de2-8e7c-8d08892b4173 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:32.644614 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eedd7b6-e91f-4799-82bc-77709f195fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:32.644614 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eedd7b6-e91f-4799-82bc-77709f195fa0 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:32.644614 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eedd7b6-e91f-4799-82bc-77709f195fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:32.644614 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eedd7b6-e91f-4799-82bc-77709f195fa0 None admin] PATH_INFO: `/v3/services` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:32.645023 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eedd7b6-e91f-4799-82bc-77709f195fa0 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:32.645023 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eedd7b6-e91f-4799-82bc-77709f195fa0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:32.645023 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eedd7b6-e91f-4799-82bc-77709f195fa0 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:32.651550 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 358/702] 10.222.0.85 () {56 vars in 1124 bytes} [Tue Jun 7 14:20:32 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:20:32.724675 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b0339a-fb86-4de2-8e7c-8d08892b4173 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:32.760812 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:32.766883 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327beaf3-c8e7-4667-b53a-5ba1a084cc19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:32.767625 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327beaf3-c8e7-4667-b53a-5ba1a084cc19 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:32.767911 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327beaf3-c8e7-4667-b53a-5ba1a084cc19 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:32.768299 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327beaf3-c8e7-4667-b53a-5ba1a084cc19 None admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:32.769303 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327beaf3-c8e7-4667-b53a-5ba1a084cc19 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:32.769620 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327beaf3-c8e7-4667-b53a-5ba1a084cc19 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:32.771232 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327beaf3-c8e7-4667-b53a-5ba1a084cc19 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:32.792762 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 345/703] 10.222.0.85 () {56 vars in 1133 bytes} [Tue Jun 7 14:20:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:20:32.798556 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eedd7b6-e91f-4799-82bc-77709f195fa0 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:32.833090 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:32.913528 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eedd7b6-e91f-4799-82bc-77709f195fa0 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:32.955430 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9dea87-831f-4fb7-bef2-cfe2bbb37b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:32.956096 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9dea87-831f-4fb7-bef2-cfe2bbb37b08 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:32.956406 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9dea87-831f-4fb7-bef2-cfe2bbb37b08 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:32.956695 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9dea87-831f-4fb7-bef2-cfe2bbb37b08 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:33.036964 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9dea87-831f-4fb7-bef2-cfe2bbb37b08 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:33.039189 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9dea87-831f-4fb7-bef2-cfe2bbb37b08 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:33.075218 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:33.081655 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e871abb-12f2-44c8-a266-e8d29c6c348d None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:33.082178 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e871abb-12f2-44c8-a266-e8d29c6c348d None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:33.082178 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e871abb-12f2-44c8-a266-e8d29c6c348d None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:33.083673 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 346/704] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:20:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 14:20:33.089791 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9272ab65-09b6-4d3b-8a3f-aa7971568d08 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:33.090372 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9272ab65-09b6-4d3b-8a3f-aa7971568d08 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:33.098586 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9272ab65-09b6-4d3b-8a3f-aa7971568d08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:33.109233 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-9272ab65-09b6-4d3b-8a3f-aa7971568d08 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:20:33.110431 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-9272ab65-09b6-4d3b-8a3f-aa7971568d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:20:33.185687 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 347/705] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:20:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:20:33.201881 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 359/706] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jun 7 14:20:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 14:20:33.207311 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9272ab65-09b6-4d3b-8a3f-aa7971568d08 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:33.266229 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9dea87-831f-4fb7-bef2-cfe2bbb37b08 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:33.372510 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9272ab65-09b6-4d3b-8a3f-aa7971568d08 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:33.395407 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9dea87-831f-4fb7-bef2-cfe2bbb37b08 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:33.396197 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:33.413971 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196d59b2-2b25-406b-9bad-039dd6a166dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:33.415272 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196d59b2-2b25-406b-9bad-039dd6a166dc None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:33.415774 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196d59b2-2b25-406b-9bad-039dd6a166dc None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:33.416244 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196d59b2-2b25-406b-9bad-039dd6a166dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:33.427579 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196d59b2-2b25-406b-9bad-039dd6a166dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:33.427770 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:33.431021 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196d59b2-2b25-406b-9bad-039dd6a166dc None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:33.433641 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f3bcaa-0c4d-489f-b2fb-8b31d19fbf16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:33.434650 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f3bcaa-0c4d-489f-b2fb-8b31d19fbf16 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:33.434778 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f3bcaa-0c4d-489f-b2fb-8b31d19fbf16 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:33.435169 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f3bcaa-0c4d-489f-b2fb-8b31d19fbf16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:33.446900 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f3bcaa-0c4d-489f-b2fb-8b31d19fbf16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:33.448293 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f3bcaa-0c4d-489f-b2fb-8b31d19fbf16 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:33.453780 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-196d59b2-2b25-406b-9bad-039dd6a166dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:33.453780 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:33.453780 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:33.453780 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:33.453780 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:33.453780 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:33.453780 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:33.453780 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:33.453780 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:33.453780 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:33.453780 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:33.453780 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:33.453780 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:20:33.453780 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:20:33.453780 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:20:33.454647 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:20:33.454647 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:33.454647 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:33.454647 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:33.454647 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:33.454647 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:33.454647 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:33.454647 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:33.454647 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:33.454647 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:33.454647 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:33.454647 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:33.454647 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:33.454647 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:33.454647 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:33.454647 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:33.454647 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:33.455524 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:20:33.455524 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:20:33.455524 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:33.455524 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:33.455524 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 348/707] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:20:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:33.456562 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94f15aa-631f-4422-81ef-28a7e48bdb6d None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:33.457443 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94f15aa-631f-4422-81ef-28a7e48bdb6d None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:33.457443 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94f15aa-631f-4422-81ef-28a7e48bdb6d None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:33.458637 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 349/708] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:20:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:20:33.465363 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94f15aa-631f-4422-81ef-28a7e48bdb6d None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:33.470321 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-24f3bcaa-0c4d-489f-b2fb-8b31d19fbf16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:33.470321 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:33.470321 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:33.470321 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:33.470321 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:33.470321 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:33.470321 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:33.470321 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:33.470321 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:33.470321 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:33.470321 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:33.470321 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:33.470321 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:20:33.470321 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:20:33.470321 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:20:33.471250 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:20:33.471250 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:33.471250 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:33.471250 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:33.471250 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:33.471250 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:33.471250 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:33.471250 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:33.471250 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:33.471250 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:33.471250 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:33.471250 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:33.471250 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:33.471250 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:33.471250 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:33.471250 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:33.471250 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:33.472200 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:20:33.472200 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:20:33.472200 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:33.472200 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:33.472200 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 360/709] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:20:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:33.473521 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f66812f-191e-4202-be50-314205c04957 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:33.473817 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f66812f-191e-4202-be50-314205c04957 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:33.474106 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f66812f-191e-4202-be50-314205c04957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:33.492786 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-2f66812f-191e-4202-be50-314205c04957 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:20:33.494003 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-2f66812f-191e-4202-be50-314205c04957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:20:33.576657 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 361/710] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:20:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:20:33.577731 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f66812f-191e-4202-be50-314205c04957 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:33.583531 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94f15aa-631f-4422-81ef-28a7e48bdb6d None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:33.627104 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3809d0ef-d2e0-4530-9fdb-3382f647f466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:33.627776 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3809d0ef-d2e0-4530-9fdb-3382f647f466 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:33.628059 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3809d0ef-d2e0-4530-9fdb-3382f647f466 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:33.628334 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3809d0ef-d2e0-4530-9fdb-3382f647f466 None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:33.629097 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3809d0ef-d2e0-4530-9fdb-3382f647f466 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:33.629400 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3809d0ef-d2e0-4530-9fdb-3382f647f466 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:33.630765 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3809d0ef-d2e0-4530-9fdb-3382f647f466 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:33.644241 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 350/711] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jun 7 14:20:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:33.645244 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3809d0ef-d2e0-4530-9fdb-3382f647f466 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:33.677670 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:33.700461 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f66812f-191e-4202-be50-314205c04957 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:33.715964 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:33.738950 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2ed669-937e-4bec-ad50-865f88ccab34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:33.739718 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2ed669-937e-4bec-ad50-865f88ccab34 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:33.740076 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2ed669-937e-4bec-ad50-865f88ccab34 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:33.740473 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2ed669-937e-4bec-ad50-865f88ccab34 None admin] PATH_INFO: `/v3/domains` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:33.741315 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2ed669-937e-4bec-ad50-865f88ccab34 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:33.741788 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2ed669-937e-4bec-ad50-865f88ccab34 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:33.743357 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2ed669-937e-4bec-ad50-865f88ccab34 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:33.751384 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3809d0ef-d2e0-4530-9fdb-3382f647f466 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:33.757199 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 362/712] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jun 7 14:20:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:33.758185 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2ed669-937e-4bec-ad50-865f88ccab34 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:33.790339 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22c7cad-24bd-40e7-9505-d1a24ecbb794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:33.791144 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22c7cad-24bd-40e7-9505-d1a24ecbb794 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:33.791426 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22c7cad-24bd-40e7-9505-d1a24ecbb794 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:33.791763 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22c7cad-24bd-40e7-9505-d1a24ecbb794 None admin] PATH_INFO: `/v3/services/network` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:33.792548 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22c7cad-24bd-40e7-9505-d1a24ecbb794 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:33.794012 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22c7cad-24bd-40e7-9505-d1a24ecbb794 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:33.808438 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-c22c7cad-24bd-40e7-9505-d1a24ecbb794 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 07 14:20:33.808438 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:33.808438 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 14:20:33.808438 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 14:20:33.808438 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 14:20:33.808438 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 14:20:33.808438 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 14:20:33.808438 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:20:33.808438 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 07 14:20:33.808438 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:33.808438 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 14:20:33.808438 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:33.808438 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:33.808438 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:33.808438 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:33.808438 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:33.808438 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:33.810730 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:33.810730 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:33.810730 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:33.810730 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:33.810730 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:33.810730 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:33.810730 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 14:20:33.810730 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 14:20:33.810730 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 14:20:33.810730 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 14:20:33.810730 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:33.810730 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:33.810730 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:33.810730 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:33.810730 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:33.810730 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:33.812802 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:33.812802 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:33.812802 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:33.812802 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:33.812802 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:33.812802 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:33.812802 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:33.812802 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:33.812802 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:33.812802 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:33.812802 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 14:20:33.812802 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:20:33.812802 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 07 14:20:33.812802 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:33.814606 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 351/713] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jun 7 14:20:33 2022] GET /identity/v3/services/network => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:33.816490 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22c7cad-24bd-40e7-9505-d1a24ecbb794 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:33.869823 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2ed669-937e-4bec-ad50-865f88ccab34 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:33.906876 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294e2836-0794-4f7f-be39-9785e7083972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:33.907799 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294e2836-0794-4f7f-be39-9785e7083972 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:33.908032 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294e2836-0794-4f7f-be39-9785e7083972 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:33.908316 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294e2836-0794-4f7f-be39-9785e7083972 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:33.915619 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294e2836-0794-4f7f-be39-9785e7083972 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:33.917408 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294e2836-0794-4f7f-be39-9785e7083972 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:33.919953 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:33.926651 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22c7cad-24bd-40e7-9505-d1a24ecbb794 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:33.933916 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-294e2836-0794-4f7f-be39-9785e7083972 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:20:33.933916 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:33.933916 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:33.933916 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:33.933916 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:33.933916 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:33.933916 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:33.933916 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:33.933916 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:33.933916 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:33.933916 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:33.933916 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:33.933916 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 14:20:33.933916 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 14:20:33.933916 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 14:20:33.944808 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 14:20:33.944808 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:33.944808 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:33.944808 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:33.944808 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:33.944808 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:33.944808 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:33.944808 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:33.944808 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:33.944808 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:33.944808 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:33.944808 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:33.944808 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:33.944808 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:33.944808 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:33.944808 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:33.944808 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:33.945920 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 14:20:33.945920 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 14:20:33.945920 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 14:20:33.945920 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 14:20:33.945920 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 14:20:33.945920 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 14:20:33.945920 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:20:33.945920 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:33.945920 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 363/714] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jun 7 14:20:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:33.945920 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294e2836-0794-4f7f-be39-9785e7083972 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:33.970024 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c547473-c4e6-4431-85a2-27ef643071d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:33.970704 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c547473-c4e6-4431-85a2-27ef643071d5 None admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:33.971087 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c547473-c4e6-4431-85a2-27ef643071d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:33.971366 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c547473-c4e6-4431-85a2-27ef643071d5 None admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:33.972186 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c547473-c4e6-4431-85a2-27ef643071d5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:33.973637 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c547473-c4e6-4431-85a2-27ef643071d5 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:33.974848 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:34.045668 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294e2836-0794-4f7f-be39-9785e7083972 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:34.087641 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71929044-b50d-46d9-a88e-be79d81376ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:34.088324 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71929044-b50d-46d9-a88e-be79d81376ec None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:34.088641 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71929044-b50d-46d9-a88e-be79d81376ec None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:34.088928 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71929044-b50d-46d9-a88e-be79d81376ec None admin] PATH_INFO: `/v3/services` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:34.089652 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71929044-b50d-46d9-a88e-be79d81376ec None admin] RBAC: Adding query filter params (name=network) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:34.089963 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71929044-b50d-46d9-a88e-be79d81376ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:34.091589 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71929044-b50d-46d9-a88e-be79d81376ec None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:34.101232 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 364/715] 10.222.0.85 () {56 vars in 1112 bytes} [Tue Jun 7 14:20:33 2022] GET /identity/v3/services?name=network => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:34.101232 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71929044-b50d-46d9-a88e-be79d81376ec None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:34.110363 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 352/716] 10.222.0.85 () {60 vars in 1132 bytes} [Tue Jun 7 14:20:33 2022] POST /identity/v3/users => generated 254 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:20:34.110363 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c547473-c4e6-4431-85a2-27ef643071d5 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:34.258979 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c547473-c4e6-4431-85a2-27ef643071d5 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:34.296404 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568a5336-4709-462f-bdc1-24971ad9c99f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:34.297494 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568a5336-4709-462f-bdc1-24971ad9c99f None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:34.298126 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568a5336-4709-462f-bdc1-24971ad9c99f None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:34.298610 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568a5336-4709-462f-bdc1-24971ad9c99f None admin] PATH_INFO: `/v3/services` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:34.299550 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568a5336-4709-462f-bdc1-24971ad9c99f None admin] RBAC: Adding query filter params (type=network) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:34.300072 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568a5336-4709-462f-bdc1-24971ad9c99f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:34.301860 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568a5336-4709-462f-bdc1-24971ad9c99f None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:34.309388 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 353/717] 10.222.0.85 () {56 vars in 1112 bytes} [Tue Jun 7 14:20:34 2022] GET /identity/v3/services?type=network => generated 124 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:34.322642 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:34.329492 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71929044-b50d-46d9-a88e-be79d81376ec None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:34.383811 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0641e9c-2245-4e0b-9c4e-4fa3a7be959d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:34.384550 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0641e9c-2245-4e0b-9c4e-4fa3a7be959d None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:34.384842 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0641e9c-2245-4e0b-9c4e-4fa3a7be959d None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:34.385121 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0641e9c-2245-4e0b-9c4e-4fa3a7be959d None admin] PATH_INFO: `/v3/projects` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:34.385928 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0641e9c-2245-4e0b-9c4e-4fa3a7be959d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:34.386242 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0641e9c-2245-4e0b-9c4e-4fa3a7be959d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:34.387711 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0641e9c-2245-4e0b-9c4e-4fa3a7be959d None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:34.410456 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 365/718] 10.222.0.85 () {56 vars in 1148 bytes} [Tue Jun 7 14:20:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:34.417340 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568a5336-4709-462f-bdc1-24971ad9c99f None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:34.435772 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:34.534417 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568a5336-4709-462f-bdc1-24971ad9c99f None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:34.572334 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5218cfe4-39e0-4d41-a1bf-54bb7b3f6c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:34.573029 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5218cfe4-39e0-4d41-a1bf-54bb7b3f6c95 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:34.573315 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5218cfe4-39e0-4d41-a1bf-54bb7b3f6c95 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:34.573594 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5218cfe4-39e0-4d41-a1bf-54bb7b3f6c95 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:34.574438 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5218cfe4-39e0-4d41-a1bf-54bb7b3f6c95 None admin] RBAC: Adding query filter params (role.id=9baff2b18f2649f7b757ff2e1f63c853, scope.project.id=724785eb74e44daf8736fd01fc0d1839, user.id=8e857db4902e42989d0d040bbc8517c8) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:34.574752 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5218cfe4-39e0-4d41-a1bf-54bb7b3f6c95 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:34.576186 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5218cfe4-39e0-4d41-a1bf-54bb7b3f6c95 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:34.582684 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 354/719] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jun 7 14:20:34 2022] GET /identity/v3/role_assignments?user.id=8e857db4902e42989d0d040bbc8517c8&scope.project.id=724785eb74e44daf8736fd01fc0d1839&role.id=9baff2b18f2649f7b757ff2e1f63c853 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:35.290871 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13dd4b5-996f-4b7e-a2f8-c133d1b2c600 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:35.291408 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13dd4b5-996f-4b7e-a2f8-c133d1b2c600 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:35.291408 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13dd4b5-996f-4b7e-a2f8-c133d1b2c600 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:35.293190 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 366/720] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:20:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:20:35.298291 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44f916e-92b2-4340-8ee0-38054f91f533 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:35.298730 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44f916e-92b2-4340-8ee0-38054f91f533 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:35.298948 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44f916e-92b2-4340-8ee0-38054f91f533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:35.325382 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-c44f916e-92b2-4340-8ee0-38054f91f533 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:20:35.326596 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-c44f916e-92b2-4340-8ee0-38054f91f533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:20:35.415355 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 355/721] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:20:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:20:35.450687 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13dd4b5-996f-4b7e-a2f8-c133d1b2c600 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:35.574784 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cec6da9-ddea-4ac7-87b4-c6ab3ef260fb None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:35.575086 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cec6da9-ddea-4ac7-87b4-c6ab3ef260fb None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:35.575274 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cec6da9-ddea-4ac7-87b4-c6ab3ef260fb None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:35.580468 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 356/722] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:20:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 14:20:35.581601 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13dd4b5-996f-4b7e-a2f8-c133d1b2c600 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:35.586396 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8979d56-5cde-41c1-ae47-4f454c0d6304 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:35.586719 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8979d56-5cde-41c1-ae47-4f454c0d6304 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:35.586965 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8979d56-5cde-41c1-ae47-4f454c0d6304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:35.621936 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99edbaff-4aed-4bbc-842c-bc04e434804e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:35.623368 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99edbaff-4aed-4bbc-842c-bc04e434804e None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:35.623858 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99edbaff-4aed-4bbc-842c-bc04e434804e None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:35.624357 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99edbaff-4aed-4bbc-842c-bc04e434804e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:35.632283 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-b8979d56-5cde-41c1-ae47-4f454c0d6304 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:20:35.633512 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-b8979d56-5cde-41c1-ae47-4f454c0d6304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:20:35.636078 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99edbaff-4aed-4bbc-842c-bc04e434804e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:20:35.636671 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:35.645071 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-99edbaff-4aed-4bbc-842c-bc04e434804e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:20:35.645071 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:35.645071 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:35.645071 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:35.645071 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:35.645071 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:35.645071 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:35.645071 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:35.645071 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:35.645071 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:35.645071 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:35.645071 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:35.645071 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:20:35.645071 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:20:35.645071 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:20:35.645071 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:20:35.646003 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:20:35.646003 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:20:35.646003 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:20:35.646003 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:20:35.646003 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:35.646003 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:35.646003 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:35.646003 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:35.646003 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:35.646003 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:35.646003 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:35.646003 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:35.646003 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:35.646003 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:35.646003 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:35.646003 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:35.646876 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:35.646876 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:35.646876 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:35.646876 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:35.646876 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:20:35.646876 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:20:35.646876 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:20:35.646876 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:20:35.646876 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:20:35.646876 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:20:35.646876 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:20:35.646876 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:35.650325 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 367/723] 10.222.0.85 () {56 vars in 1096 bytes} [Tue Jun 7 14:20:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:35.657118 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99edbaff-4aed-4bbc-842c-bc04e434804e None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:35.692772 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:35.766634 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99edbaff-4aed-4bbc-842c-bc04e434804e None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:35.823579 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fee246-a45d-4a14-b0e0-5b8b33fbe59f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:35.824718 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fee246-a45d-4a14-b0e0-5b8b33fbe59f None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:35.825272 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fee246-a45d-4a14-b0e0-5b8b33fbe59f None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:35.825750 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fee246-a45d-4a14-b0e0-5b8b33fbe59f None admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:35.826718 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fee246-a45d-4a14-b0e0-5b8b33fbe59f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:35.827244 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fee246-a45d-4a14-b0e0-5b8b33fbe59f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:35.829107 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fee246-a45d-4a14-b0e0-5b8b33fbe59f None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:35.854608 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 368/724] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jun 7 14:20:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:20:35.857051 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fee246-a45d-4a14-b0e0-5b8b33fbe59f None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:35.862065 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 357/725] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:20:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:20:35.883334 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8979d56-5cde-41c1-ae47-4f454c0d6304 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:36.033711 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:36.059636 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8979d56-5cde-41c1-ae47-4f454c0d6304 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:36.079026 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fee246-a45d-4a14-b0e0-5b8b33fbe59f None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:36.118245 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a36f961-6ed7-4369-89ae-d09a2221cbc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:36.119317 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a36f961-6ed7-4369-89ae-d09a2221cbc3 None admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:36.120094 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a36f961-6ed7-4369-89ae-d09a2221cbc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:36.120610 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a36f961-6ed7-4369-89ae-d09a2221cbc3 None admin] PATH_INFO: `/v3/services` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:36.121580 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a36f961-6ed7-4369-89ae-d09a2221cbc3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:36.123154 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a36f961-6ed7-4369-89ae-d09a2221cbc3 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:36.146135 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5640e64a-de61-4932-b027-d87cdf1f1c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:36.153407 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5640e64a-de61-4932-b027-d87cdf1f1c78 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:36.153407 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5640e64a-de61-4932-b027-d87cdf1f1c78 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:36.153407 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5640e64a-de61-4932-b027-d87cdf1f1c78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:36.159214 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5640e64a-de61-4932-b027-d87cdf1f1c78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:36.160927 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5640e64a-de61-4932-b027-d87cdf1f1c78 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:36.180382 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-5640e64a-de61-4932-b027-d87cdf1f1c78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:36.180382 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:36.180382 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:36.180382 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:36.180382 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:36.180382 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:36.180382 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:36.180382 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:36.180382 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:36.180382 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:36.180382 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:36.180382 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:36.180382 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:20:36.180382 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:20:36.180382 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:20:36.181306 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:20:36.181306 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:36.181306 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:36.181306 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:36.181306 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:36.181306 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:36.181306 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:36.181306 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:36.181306 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:36.181306 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:36.181306 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:36.181306 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:36.181306 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:36.181306 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:36.181306 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:36.181306 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:36.181306 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:36.182212 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:20:36.182212 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:20:36.182212 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:36.182212 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:36.184361 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 369/726] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:20:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:36.190079 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5640e64a-de61-4932-b027-d87cdf1f1c78 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:36.266727 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:36.274091 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 358/727] 10.222.0.85 () {60 vars in 1141 bytes} [Tue Jun 7 14:20:35 2022] POST /identity/v3/services => generated 251 bytes in 391 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:20:36.308271 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5640e64a-de61-4932-b027-d87cdf1f1c78 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:36.340761 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:36.357913 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be593986-f61d-428c-ac48-611c5bdde3b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:36.358951 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be593986-f61d-428c-ac48-611c5bdde3b1 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:36.362172 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be593986-f61d-428c-ac48-611c5bdde3b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:36.362172 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be593986-f61d-428c-ac48-611c5bdde3b1 None admin] PATH_INFO: `/v3/domains` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:36.362172 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be593986-f61d-428c-ac48-611c5bdde3b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:36.362172 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be593986-f61d-428c-ac48-611c5bdde3b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:36.362790 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be593986-f61d-428c-ac48-611c5bdde3b1 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:36.382362 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 370/728] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jun 7 14:20:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:36.390089 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a36f961-6ed7-4369-89ae-d09a2221cbc3 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:36.532744 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a36f961-6ed7-4369-89ae-d09a2221cbc3 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:36.567859 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9599c011-d261-40da-bbc7-6200dd1971ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:36.569677 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9599c011-d261-40da-bbc7-6200dd1971ee None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:36.570257 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9599c011-d261-40da-bbc7-6200dd1971ee None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:36.570733 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9599c011-d261-40da-bbc7-6200dd1971ee None admin] PATH_INFO: `/v3/users/swift` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:36.579500 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9599c011-d261-40da-bbc7-6200dd1971ee None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:36.581284 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9599c011-d261-40da-bbc7-6200dd1971ee None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:36.597858 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-9599c011-d261-40da-bbc7-6200dd1971ee None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 07 14:20:36.597858 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:36.597858 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:36.597858 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:36.597858 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:36.597858 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:36.597858 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:36.597858 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:36.597858 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:36.597858 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:36.597858 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:36.597858 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:36.597858 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 14:20:36.597858 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 14:20:36.597858 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 14:20:36.597858 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 14:20:36.598874 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:36.598874 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:36.598874 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 14:20:36.598874 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:20:36.598874 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 14:20:36.598874 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:20:36.598874 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:36.598874 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:36.598874 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:36.598874 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:36.598874 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:36.598874 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:36.598874 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:36.598874 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:36.598874 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:36.598874 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:36.598874 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:36.598874 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:36.599824 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:36.599824 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:36.599824 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 14:20:36.599824 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 14:20:36.599824 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 14:20:36.599824 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 14:20:36.599824 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 14:20:36.599824 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 14:20:36.599824 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 07 14:20:36.599824 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:36.602499 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 359/729] 10.222.0.85 () {56 vars in 1096 bytes} [Tue Jun 7 14:20:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:36.608193 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be593986-f61d-428c-ac48-611c5bdde3b1 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:36.719081 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be593986-f61d-428c-ac48-611c5bdde3b1 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:36.757485 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198a7901-7512-4aad-8cc7-0c42528978ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:36.758247 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198a7901-7512-4aad-8cc7-0c42528978ad None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:36.758539 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198a7901-7512-4aad-8cc7-0c42528978ad None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:36.758823 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198a7901-7512-4aad-8cc7-0c42528978ad None admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:36.764136 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198a7901-7512-4aad-8cc7-0c42528978ad None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:36.764136 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198a7901-7512-4aad-8cc7-0c42528978ad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:36.765786 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198a7901-7512-4aad-8cc7-0c42528978ad None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:36.794041 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 371/730] 10.222.0.85 () {56 vars in 1135 bytes} [Tue Jun 7 14:20:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:36.799908 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9599c011-d261-40da-bbc7-6200dd1971ee None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:36.897558 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:36.961152 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9599c011-d261-40da-bbc7-6200dd1971ee None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:37.008634 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95f1f19-188b-4eb7-a774-1c95e4a12963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:37.009299 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95f1f19-188b-4eb7-a774-1c95e4a12963 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:37.009739 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95f1f19-188b-4eb7-a774-1c95e4a12963 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:37.009917 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95f1f19-188b-4eb7-a774-1c95e4a12963 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:37.107855 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95f1f19-188b-4eb7-a774-1c95e4a12963 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:37.109361 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95f1f19-188b-4eb7-a774-1c95e4a12963 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:37.165759 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467e5ff5-9d1a-46c2-9f84-dfe9bcade5b0 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:37.166060 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467e5ff5-9d1a-46c2-9f84-dfe9bcade5b0 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:37.166246 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467e5ff5-9d1a-46c2-9f84-dfe9bcade5b0 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:37.167635 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 372/731] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:20:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:20:37.173074 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2256e5-0bf6-4f59-8763-6223911491b9 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:37.173366 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2256e5-0bf6-4f59-8763-6223911491b9 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:37.173646 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2256e5-0bf6-4f59-8763-6223911491b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:37.193075 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-4b2256e5-0bf6-4f59-8763-6223911491b9 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:20:37.194900 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-4b2256e5-0bf6-4f59-8763-6223911491b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:20:37.215553 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:37.245524 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:37.267807 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 360/732] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jun 7 14:20:36 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 14:20:37.276398 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c8ac64-4364-4608-beaf-ceaeebeaffbc None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:37.276398 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c8ac64-4364-4608-beaf-ceaeebeaffbc None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:37.276398 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c8ac64-4364-4608-beaf-ceaeebeaffbc None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:37.276398 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 361/733] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:20:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:20:37.276398 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60534ee-e93b-46d1-ae3f-34a9b7997e8e None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:37.276398 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60534ee-e93b-46d1-ae3f-34a9b7997e8e None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:37.276882 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60534ee-e93b-46d1-ae3f-34a9b7997e8e None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:37.276882 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 362/734] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:20:37 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:20:37.276882 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60534ee-e93b-46d1-ae3f-34a9b7997e8e None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:37.281411 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 373/735] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:20:37 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:20:37.282828 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218b29e4-1b50-44ce-bc55-17a4730945cf None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:37.283123 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218b29e4-1b50-44ce-bc55-17a4730945cf None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:37.283400 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218b29e4-1b50-44ce-bc55-17a4730945cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:37.306494 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-218b29e4-1b50-44ce-bc55-17a4730945cf None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:20:37.307697 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-218b29e4-1b50-44ce-bc55-17a4730945cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:20:37.381804 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 374/736] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:20:37 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:20:37.386437 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2118c12e-170f-4f37-bb31-55e8a5f32ef8 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:37.387187 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2118c12e-170f-4f37-bb31-55e8a5f32ef8 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:37.387670 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2118c12e-170f-4f37-bb31-55e8a5f32ef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:37.389709 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60534ee-e93b-46d1-ae3f-34a9b7997e8e None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:37.409904 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-2118c12e-170f-4f37-bb31-55e8a5f32ef8 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:20:37.411088 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-2118c12e-170f-4f37-bb31-55e8a5f32ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:20:37.430083 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0481738a-aba6-445c-9591-04a61dd94c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:37.430794 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0481738a-aba6-445c-9591-04a61dd94c83 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:37.431257 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0481738a-aba6-445c-9591-04a61dd94c83 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:37.431685 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0481738a-aba6-445c-9591-04a61dd94c83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:37.442461 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0481738a-aba6-445c-9591-04a61dd94c83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:37.444136 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0481738a-aba6-445c-9591-04a61dd94c83 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:37.463046 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-0481738a-aba6-445c-9591-04a61dd94c83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:37.463046 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:37.463046 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:37.463046 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:37.463046 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:37.463046 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:37.463046 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:37.463046 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:37.463046 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:37.463046 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:37.463046 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:37.463046 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:37.463046 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:20:37.463046 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:20:37.463046 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:20:37.463964 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:20:37.463964 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:37.463964 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:37.463964 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:37.463964 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:37.463964 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:37.463964 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:37.463964 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:37.463964 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:37.463964 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:37.463964 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:37.463964 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:37.463964 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:37.463964 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:37.463964 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:37.463964 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:37.463964 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:37.464890 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:20:37.464890 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:20:37.464890 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:37.464890 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:37.464890 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 363/737] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:20:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:37.465161 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0481738a-aba6-445c-9591-04a61dd94c83 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:37.484006 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:37.492415 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 375/738] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:20:37 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 14:20:37.492415 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2118c12e-170f-4f37-bb31-55e8a5f32ef8 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:37.534288 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:37.587728 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0481738a-aba6-445c-9591-04a61dd94c83 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:37.601731 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2118c12e-170f-4f37-bb31-55e8a5f32ef8 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:37.624433 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ebbe24-397f-4a7e-baa4-131032a41760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:37.625099 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ebbe24-397f-4a7e-baa4-131032a41760 None admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:37.625384 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ebbe24-397f-4a7e-baa4-131032a41760 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:37.625660 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ebbe24-397f-4a7e-baa4-131032a41760 None admin] PATH_INFO: `/v3/services` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:37.626384 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ebbe24-397f-4a7e-baa4-131032a41760 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:37.627748 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ebbe24-397f-4a7e-baa4-131032a41760 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:37.637690 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a12c0a-1745-4528-8c92-cf57c948699f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:37.638484 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a12c0a-1745-4528-8c92-cf57c948699f None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:37.638776 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a12c0a-1745-4528-8c92-cf57c948699f None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:37.639055 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a12c0a-1745-4528-8c92-cf57c948699f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:37.650311 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a12c0a-1745-4528-8c92-cf57c948699f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:20:37.658109 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-68a12c0a-1745-4528-8c92-cf57c948699f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:20:37.658109 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:37.658109 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:37.658109 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:37.658109 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:37.658109 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:37.658109 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:37.658109 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:37.658109 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:37.658109 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:37.658109 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:37.658109 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:37.658109 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:20:37.658109 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:20:37.658109 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:20:37.658109 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:20:37.659114 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:20:37.659114 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:20:37.659114 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:20:37.659114 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:20:37.659114 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:37.659114 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:37.659114 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:37.659114 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:37.659114 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:37.659114 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:37.659114 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:37.659114 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:37.659114 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:37.659114 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:37.659114 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:37.659114 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:37.659980 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:37.659980 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:37.659980 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:37.659980 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:37.659980 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:20:37.659980 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:20:37.659980 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:20:37.659980 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:20:37.659980 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:20:37.659980 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:20:37.659980 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:20:37.659980 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:37.659980 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 376/739] 10.222.0.85 () {56 vars in 1102 bytes} [Tue Jun 7 14:20:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:37.661185 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a12c0a-1745-4528-8c92-cf57c948699f None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:37.705524 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 364/740] 10.222.0.85 () {60 vars in 1141 bytes} [Tue Jun 7 14:20:37 2022] POST /identity/v3/services => generated 240 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:20:37.706712 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ebbe24-397f-4a7e-baa4-131032a41760 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:37.789454 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:37.796233 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:37.799719 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a12c0a-1745-4528-8c92-cf57c948699f None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:37.848740 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ebbe24-397f-4a7e-baa4-131032a41760 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:37.856009 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2f17fa-1f78-4ea4-ba9e-f8668ae3287e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:37.856726 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2f17fa-1f78-4ea4-ba9e-f8668ae3287e None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:37.857213 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2f17fa-1f78-4ea4-ba9e-f8668ae3287e None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:37.857494 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2f17fa-1f78-4ea4-ba9e-f8668ae3287e None admin] PATH_INFO: `/v3/domains` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:37.859357 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2f17fa-1f78-4ea4-ba9e-f8668ae3287e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:37.859718 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2f17fa-1f78-4ea4-ba9e-f8668ae3287e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:37.861277 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2f17fa-1f78-4ea4-ba9e-f8668ae3287e None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:37.878050 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 377/741] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jun 7 14:20:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:37.878935 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2f17fa-1f78-4ea4-ba9e-f8668ae3287e None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:37.904005 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d41b20d-6ecd-4526-bfb6-c923344e7f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:37.904862 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d41b20d-6ecd-4526-bfb6-c923344e7f17 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:37.905212 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d41b20d-6ecd-4526-bfb6-c923344e7f17 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:37.905497 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d41b20d-6ecd-4526-bfb6-c923344e7f17 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:37.921767 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d41b20d-6ecd-4526-bfb6-c923344e7f17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:20:37.929754 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-3d41b20d-6ecd-4526-bfb6-c923344e7f17 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:20:37.929754 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:37.929754 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:37.929754 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:37.929754 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:37.929754 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:37.929754 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:37.929754 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:37.929754 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:37.929754 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:37.929754 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:37.929754 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:37.929754 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:20:37.929754 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:20:37.929754 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:20:37.929754 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:20:37.930692 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:20:37.930692 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:20:37.930692 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:20:37.930692 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:20:37.930692 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:37.930692 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:37.930692 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:37.930692 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:37.930692 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:37.930692 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:37.930692 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:37.930692 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:37.930692 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:37.930692 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:37.930692 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:37.930692 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:37.931533 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:37.931533 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:37.931533 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:37.931533 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:37.931533 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:20:37.931533 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:20:37.931533 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:20:37.931533 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:20:37.931533 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:20:37.931533 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:20:37.931533 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:20:37.931533 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:37.931533 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 365/742] 10.222.0.85 () {56 vars in 1096 bytes} [Tue Jun 7 14:20:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:37.932227 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d41b20d-6ecd-4526-bfb6-c923344e7f17 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:38.012930 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2f17fa-1f78-4ea4-ba9e-f8668ae3287e None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:38.058262 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968e583b-dd97-4c4d-bba4-7b95d3fad3f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:38.058833 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968e583b-dd97-4c4d-bba4-7b95d3fad3f6 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:38.059122 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968e583b-dd97-4c4d-bba4-7b95d3fad3f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:38.059404 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968e583b-dd97-4c4d-bba4-7b95d3fad3f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:38.060163 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968e583b-dd97-4c4d-bba4-7b95d3fad3f6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:38.060502 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968e583b-dd97-4c4d-bba4-7b95d3fad3f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:38.061898 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968e583b-dd97-4c4d-bba4-7b95d3fad3f6 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:38.062980 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d41b20d-6ecd-4526-bfb6-c923344e7f17 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:38.071092 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 378/743] 10.222.0.85 () {56 vars in 1103 bytes} [Tue Jun 7 14:20:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:38.076306 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968e583b-dd97-4c4d-bba4-7b95d3fad3f6 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:38.099254 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:38.111294 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dbfdef-6a5a-407b-a408-e218823da2d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:38.112102 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dbfdef-6a5a-407b-a408-e218823da2d0 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:38.112650 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dbfdef-6a5a-407b-a408-e218823da2d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:38.112942 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dbfdef-6a5a-407b-a408-e218823da2d0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:38.116797 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:38.133938 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dbfdef-6a5a-407b-a408-e218823da2d0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:38.135422 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dbfdef-6a5a-407b-a408-e218823da2d0 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:38.160816 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-c9dbfdef-6a5a-407b-a408-e218823da2d0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:20:38.160816 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:38.160816 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:38.160816 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:38.160816 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:38.160816 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:38.160816 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:38.160816 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:38.160816 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:38.160816 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:38.160816 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:38.160816 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:38.160816 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 14:20:38.160816 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 14:20:38.160816 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 14:20:38.161827 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 14:20:38.161827 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:38.161827 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:38.161827 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:38.161827 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:38.161827 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:38.161827 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:38.161827 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:38.161827 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:38.161827 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:38.161827 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:38.161827 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:38.161827 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:38.161827 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:38.161827 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:38.161827 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:38.161827 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:38.163732 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 14:20:38.163732 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 14:20:38.163732 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 14:20:38.163732 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 14:20:38.163732 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 14:20:38.163732 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 14:20:38.163732 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:20:38.163732 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:38.167731 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 366/744] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jun 7 14:20:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:38.168567 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dbfdef-6a5a-407b-a408-e218823da2d0 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:38.223423 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968e583b-dd97-4c4d-bba4-7b95d3fad3f6 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:38.262848 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc29aba-49a3-42c0-932e-ce48948af3ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:38.263551 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc29aba-49a3-42c0-932e-ce48948af3ed None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:38.263833 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc29aba-49a3-42c0-932e-ce48948af3ed None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:38.264107 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc29aba-49a3-42c0-932e-ce48948af3ed None admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:38.265279 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc29aba-49a3-42c0-932e-ce48948af3ed None admin] RBAC: Adding query filter params (name=admin) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:38.265279 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc29aba-49a3-42c0-932e-ce48948af3ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:38.266696 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc29aba-49a3-42c0-932e-ce48948af3ed None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:38.282507 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 379/745] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jun 7 14:20:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:38.284069 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc29aba-49a3-42c0-932e-ce48948af3ed None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:38.297600 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dbfdef-6a5a-407b-a408-e218823da2d0 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:38.342572 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359c5496-abbb-4d2a-8b30-91edb707c6cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:38.343730 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359c5496-abbb-4d2a-8b30-91edb707c6cf None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:38.344630 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359c5496-abbb-4d2a-8b30-91edb707c6cf None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:38.345090 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359c5496-abbb-4d2a-8b30-91edb707c6cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:38.359631 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359c5496-abbb-4d2a-8b30-91edb707c6cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:38.361835 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359c5496-abbb-4d2a-8b30-91edb707c6cf None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:38.371027 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:38.385551 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-359c5496-abbb-4d2a-8b30-91edb707c6cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:38.385551 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:38.385551 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:38.385551 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:38.385551 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:38.385551 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:38.385551 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:38.385551 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:38.385551 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:38.385551 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:38.385551 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:38.385551 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:38.385551 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:20:38.385551 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:20:38.385551 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:20:38.386464 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:20:38.386464 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:38.386464 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:38.386464 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:38.386464 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:38.386464 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:38.386464 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:38.386464 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:38.386464 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:38.386464 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:38.386464 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:38.386464 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:38.386464 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:38.386464 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:38.386464 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:38.386464 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:38.386464 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:38.387348 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:20:38.387348 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:20:38.387348 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:38.387348 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:38.390051 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 367/746] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:20:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:38.391267 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359c5496-abbb-4d2a-8b30-91edb707c6cf None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:38.403893 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc29aba-49a3-42c0-932e-ce48948af3ed None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:38.437256 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:38.450685 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d395fd6-11a9-434d-80cb-350d96e1c075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:38.452332 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d395fd6-11a9-434d-80cb-350d96e1c075 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:38.452820 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d395fd6-11a9-434d-80cb-350d96e1c075 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:38.453386 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d395fd6-11a9-434d-80cb-350d96e1c075 None admin] PATH_INFO: `/v3/projects` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:38.454619 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d395fd6-11a9-434d-80cb-350d96e1c075 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:38.455211 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d395fd6-11a9-434d-80cb-350d96e1c075 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:38.457483 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d395fd6-11a9-434d-80cb-350d96e1c075 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:38.476008 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 380/747] 10.222.0.85 () {56 vars in 1148 bytes} [Tue Jun 7 14:20:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:38.477259 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d395fd6-11a9-434d-80cb-350d96e1c075 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:38.511282 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359c5496-abbb-4d2a-8b30-91edb707c6cf None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:38.550778 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8c9272-749b-41e3-9eed-8ebf05327b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:38.551855 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8c9272-749b-41e3-9eed-8ebf05327b7b None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:38.552214 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8c9272-749b-41e3-9eed-8ebf05327b7b None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:38.552585 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8c9272-749b-41e3-9eed-8ebf05327b7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:38.564231 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8c9272-749b-41e3-9eed-8ebf05327b7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:38.566008 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8c9272-749b-41e3-9eed-8ebf05327b7b None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:38.585996 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-ca8c9272-749b-41e3-9eed-8ebf05327b7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:38.585996 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:38.585996 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:38.585996 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:38.585996 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:38.585996 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:38.585996 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:38.585996 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:38.585996 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:38.585996 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:38.585996 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:38.585996 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:38.585996 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:20:38.585996 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:20:38.585996 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:20:38.587122 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:20:38.587122 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:38.587122 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:38.587122 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:38.587122 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:38.587122 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:38.587122 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:38.587122 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:38.587122 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:38.587122 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:38.587122 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:38.587122 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:38.587122 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:38.587122 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:38.587122 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:38.587122 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:38.587122 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:38.588201 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:20:38.588201 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:20:38.588201 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:38.588201 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:38.594001 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 368/748] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:20:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:38.594001 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8c9272-749b-41e3-9eed-8ebf05327b7b None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:38.616450 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d395fd6-11a9-434d-80cb-350d96e1c075 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:38.662406 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f348bf1-927a-4166-b906-f456c1489db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:38.664651 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f348bf1-927a-4166-b906-f456c1489db8 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:38.665157 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f348bf1-927a-4166-b906-f456c1489db8 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:38.665630 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f348bf1-927a-4166-b906-f456c1489db8 None admin] PATH_INFO: `/v3/domains` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:38.666558 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f348bf1-927a-4166-b906-f456c1489db8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:38.667237 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f348bf1-927a-4166-b906-f456c1489db8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:38.669684 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f348bf1-927a-4166-b906-f456c1489db8 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:38.673563 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:38.685174 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 381/749] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jun 7 14:20:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:38.686605 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f348bf1-927a-4166-b906-f456c1489db8 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:38.698974 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:38.709993 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8c9272-749b-41e3-9eed-8ebf05327b7b None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:38.751233 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3429904-b2ce-4f3d-95ce-9b67bcd50344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:38.751882 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3429904-b2ce-4f3d-95ce-9b67bcd50344 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:38.752170 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3429904-b2ce-4f3d-95ce-9b67bcd50344 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:38.752546 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3429904-b2ce-4f3d-95ce-9b67bcd50344 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:38.753446 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3429904-b2ce-4f3d-95ce-9b67bcd50344 None admin] RBAC: Adding query filter params (role.id=9baff2b18f2649f7b757ff2e1f63c853, scope.project.id=724785eb74e44daf8736fd01fc0d1839, user.id=35aa67025f66459da0c0fc3cbfd45430) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:38.753766 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3429904-b2ce-4f3d-95ce-9b67bcd50344 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:38.755414 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3429904-b2ce-4f3d-95ce-9b67bcd50344 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:38.768737 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 369/750] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jun 7 14:20:38 2022] GET /identity/v3/role_assignments?user.id=35aa67025f66459da0c0fc3cbfd45430&scope.project.id=724785eb74e44daf8736fd01fc0d1839&role.id=9baff2b18f2649f7b757ff2e1f63c853 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:38.769896 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3429904-b2ce-4f3d-95ce-9b67bcd50344 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:38.816983 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f348bf1-927a-4166-b906-f456c1489db8 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:38.860148 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c9a93c-8d73-432a-a160-01650bf0f731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:38.861154 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c9a93c-8d73-432a-a160-01650bf0f731 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:38.861615 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c9a93c-8d73-432a-a160-01650bf0f731 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:38.862108 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c9a93c-8d73-432a-a160-01650bf0f731 None admin] PATH_INFO: `/v3/domains` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:38.863508 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c9a93c-8d73-432a-a160-01650bf0f731 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:38.863999 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c9a93c-8d73-432a-a160-01650bf0f731 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:38.865594 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c9a93c-8d73-432a-a160-01650bf0f731 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:38.886806 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 382/751] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jun 7 14:20:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:38.889883 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c9a93c-8d73-432a-a160-01650bf0f731 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:39.002862 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3429904-b2ce-4f3d-95ce-9b67bcd50344 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:39.015102 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:39.067040 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623dd489-bc6b-4a36-b494-1ca6bc2e7043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:39.067954 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623dd489-bc6b-4a36-b494-1ca6bc2e7043 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:39.068458 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623dd489-bc6b-4a36-b494-1ca6bc2e7043 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:39.068567 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623dd489-bc6b-4a36-b494-1ca6bc2e7043 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:39.076831 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623dd489-bc6b-4a36-b494-1ca6bc2e7043 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:39.078417 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623dd489-bc6b-4a36-b494-1ca6bc2e7043 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:39.086186 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c9a93c-8d73-432a-a160-01650bf0f731 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:39.098422 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-623dd489-bc6b-4a36-b494-1ca6bc2e7043 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 14:20:39.098422 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:39.098422 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:39.098422 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:39.098422 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:39.098422 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:39.098422 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:39.098422 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:39.098422 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:39.098422 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:39.098422 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:39.098422 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:39.098422 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 14:20:39.098422 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 14:20:39.098422 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 14:20:39.098422 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 14:20:39.099416 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:39.099416 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:39.099416 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 14:20:39.099416 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:20:39.099416 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 14:20:39.099416 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:20:39.099416 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:39.099416 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:39.099416 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:39.099416 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:39.099416 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:39.099416 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:39.099416 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:39.099416 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:39.099416 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:39.099416 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:39.099416 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:39.099416 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:39.100443 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:39.100443 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:39.100443 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 14:20:39.100443 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 14:20:39.100443 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 14:20:39.100443 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 14:20:39.100443 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 14:20:39.100443 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 14:20:39.100443 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 14:20:39.100443 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:39.100443 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 370/752] 10.222.0.85 () {56 vars in 1117 bytes} [Tue Jun 7 14:20:38 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:39.105765 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623dd489-bc6b-4a36-b494-1ca6bc2e7043 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:39.130971 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f518bca6-525c-41d4-b8eb-5e612fa8868c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:39.133457 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f518bca6-525c-41d4-b8eb-5e612fa8868c None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:39.134050 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:39.134900 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f518bca6-525c-41d4-b8eb-5e612fa8868c None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:39.136172 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f518bca6-525c-41d4-b8eb-5e612fa8868c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:39.153629 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f518bca6-525c-41d4-b8eb-5e612fa8868c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:39.155114 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f518bca6-525c-41d4-b8eb-5e612fa8868c None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:39.181287 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-f518bca6-525c-41d4-b8eb-5e612fa8868c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 07 14:20:39.181287 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:39.181287 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:39.181287 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:39.181287 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:39.181287 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:39.181287 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:39.181287 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:39.181287 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:39.181287 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:39.181287 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:39.181287 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:39.181287 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 14:20:39.181287 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 14:20:39.181287 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 14:20:39.181287 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 14:20:39.182283 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:39.182283 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:39.182283 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 14:20:39.182283 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:20:39.182283 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 14:20:39.182283 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:20:39.182283 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:39.182283 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:39.182283 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:39.182283 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:39.182283 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:39.182283 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:39.182283 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:39.182283 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:39.182283 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:39.182283 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:39.182283 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:39.182283 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:39.183207 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:39.183207 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:39.183207 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 14:20:39.183207 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 14:20:39.183207 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 14:20:39.183207 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 14:20:39.183207 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 14:20:39.183207 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 14:20:39.183207 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 07 14:20:39.183207 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:39.188547 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 383/753] 10.222.0.85 () {56 vars in 1093 bytes} [Tue Jun 7 14:20:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:20:39.200291 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f518bca6-525c-41d4-b8eb-5e612fa8868c None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:39.239780 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623dd489-bc6b-4a36-b494-1ca6bc2e7043 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:39.285649 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0ecdc0-d169-47e2-94ac-353ba60af090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:39.287228 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0ecdc0-d169-47e2-94ac-353ba60af090 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:39.287721 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0ecdc0-d169-47e2-94ac-353ba60af090 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:39.288178 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0ecdc0-d169-47e2-94ac-353ba60af090 None admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:39.289685 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0ecdc0-d169-47e2-94ac-353ba60af090 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:39.290589 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0ecdc0-d169-47e2-94ac-353ba60af090 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:39.292599 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0ecdc0-d169-47e2-94ac-353ba60af090 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:39.315614 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f518bca6-525c-41d4-b8eb-5e612fa8868c None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:39.326539 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 371/754] 10.222.0.85 () {56 vars in 1149 bytes} [Tue Jun 7 14:20:39 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:20:39.328730 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe92656-1662-43f4-bbe6-c0291381683b None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:39.329235 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe92656-1662-43f4-bbe6-c0291381683b None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:39.330119 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe92656-1662-43f4-bbe6-c0291381683b None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:39.332453 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 372/755] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:20:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:20:39.333459 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe92656-1662-43f4-bbe6-c0291381683b None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:39.340627 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:39.358549 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f999e99-d40e-47a9-84a8-7e4c145b84f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:39.359252 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f999e99-d40e-47a9-84a8-7e4c145b84f0 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:39.359512 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f999e99-d40e-47a9-84a8-7e4c145b84f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:39.359791 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f999e99-d40e-47a9-84a8-7e4c145b84f0 None admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:39.360665 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f999e99-d40e-47a9-84a8-7e4c145b84f0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:39.360989 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f999e99-d40e-47a9-84a8-7e4c145b84f0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:39.362433 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f999e99-d40e-47a9-84a8-7e4c145b84f0 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:39.387293 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 384/756] 10.222.0.85 () {56 vars in 1133 bytes} [Tue Jun 7 14:20:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:39.388716 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f47851-0f43-41ae-bb32-c5a64e5a6508 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:39.389029 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f47851-0f43-41ae-bb32-c5a64e5a6508 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:39.389296 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f47851-0f43-41ae-bb32-c5a64e5a6508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:39.407878 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-c9f47851-0f43-41ae-bb32-c5a64e5a6508 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:20:39.410039 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-c9f47851-0f43-41ae-bb32-c5a64e5a6508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:20:39.430431 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:39.472809 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe92656-1662-43f4-bbe6-c0291381683b None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:39.500711 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 385/757] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:20:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:20:39.502234 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f47851-0f43-41ae-bb32-c5a64e5a6508 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:39.524772 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13220a29-d9aa-40d3-96ff-08abaff892cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:39.526526 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13220a29-d9aa-40d3-96ff-08abaff892cc None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:39.526819 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13220a29-d9aa-40d3-96ff-08abaff892cc None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:39.527103 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13220a29-d9aa-40d3-96ff-08abaff892cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:39.623747 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f47851-0f43-41ae-bb32-c5a64e5a6508 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:39.626068 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13220a29-d9aa-40d3-96ff-08abaff892cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:39.627736 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:39.629184 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13220a29-d9aa-40d3-96ff-08abaff892cc None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:39.661721 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d4df9c-789b-4438-8995-f3e8333418bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:39.663723 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d4df9c-789b-4438-8995-f3e8333418bb None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:39.664222 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d4df9c-789b-4438-8995-f3e8333418bb None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:39.664823 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d4df9c-789b-4438-8995-f3e8333418bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:39.676272 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d4df9c-789b-4438-8995-f3e8333418bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:39.678065 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d4df9c-789b-4438-8995-f3e8333418bb None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:39.704910 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-52d4df9c-789b-4438-8995-f3e8333418bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:39.704910 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:39.704910 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:39.704910 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:39.704910 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:39.704910 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:39.704910 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:39.704910 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:39.704910 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:39.704910 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:39.704910 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:39.704910 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:39.704910 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:20:39.704910 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:20:39.704910 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:20:39.705849 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:20:39.705849 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:39.705849 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:39.705849 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:39.705849 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:39.705849 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:39.705849 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:39.705849 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:39.705849 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:39.705849 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:39.705849 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:39.705849 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:39.705849 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:39.705849 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:39.705849 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:39.705849 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:39.705849 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:39.706735 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:20:39.706735 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:20:39.706735 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:39.706735 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:39.708520 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 386/758] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:20:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:39.709566 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d4df9c-789b-4438-8995-f3e8333418bb None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:39.723585 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:39.774122 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 373/759] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jun 7 14:20:39 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 14:20:39.774122 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13220a29-d9aa-40d3-96ff-08abaff892cc None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:39.837460 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d4df9c-789b-4438-8995-f3e8333418bb None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:39.876269 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb6c3b8-015f-4eda-9734-1b2abd696a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:39.877505 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb6c3b8-015f-4eda-9734-1b2abd696a4d None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:39.878095 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb6c3b8-015f-4eda-9734-1b2abd696a4d None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:39.878570 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb6c3b8-015f-4eda-9734-1b2abd696a4d None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:39.879442 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb6c3b8-015f-4eda-9734-1b2abd696a4d None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:39.881142 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb6c3b8-015f-4eda-9734-1b2abd696a4d None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:39.894503 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-3cb6c3b8-015f-4eda-9734-1b2abd696a4d None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 07 14:20:39.894503 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:39.894503 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 14:20:39.894503 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 14:20:39.894503 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 14:20:39.894503 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 14:20:39.894503 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 14:20:39.894503 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:20:39.894503 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 07 14:20:39.894503 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:39.894503 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 14:20:39.894503 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:39.894503 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:39.894503 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:39.894503 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:39.894503 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:39.895435 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:39.895435 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:39.895435 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:39.895435 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:39.895435 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:39.895435 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:39.895435 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:39.895435 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 14:20:39.895435 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 14:20:39.895435 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 14:20:39.895435 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 14:20:39.895435 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:39.895435 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:39.895435 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:39.895435 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:39.895435 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:39.896265 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:39.896265 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:39.896265 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:39.896265 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:39.896265 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:39.896265 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:39.896265 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:39.896265 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:39.896265 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:39.896265 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:39.896265 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:39.896265 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 14:20:39.896265 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:20:39.896265 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 07 14:20:39.896265 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:39.899780 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 387/760] 10.222.0.85 () {56 vars in 1129 bytes} [Tue Jun 7 14:20:39 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:39.900872 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb6c3b8-015f-4eda-9734-1b2abd696a4d None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:39.907196 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13220a29-d9aa-40d3-96ff-08abaff892cc None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:39.939523 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:39.949372 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8882ef01-ffdb-4458-af04-aefbe9f8e2ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:39.958230 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8882ef01-ffdb-4458-af04-aefbe9f8e2ff None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:39.958230 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8882ef01-ffdb-4458-af04-aefbe9f8e2ff None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:39.958230 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8882ef01-ffdb-4458-af04-aefbe9f8e2ff None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:39.958230 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8882ef01-ffdb-4458-af04-aefbe9f8e2ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:39.958230 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8882ef01-ffdb-4458-af04-aefbe9f8e2ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:39.958230 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8882ef01-ffdb-4458-af04-aefbe9f8e2ff None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:39.978632 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 374/761] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jun 7 14:20:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:39.978632 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8882ef01-ffdb-4458-af04-aefbe9f8e2ff None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:40.017242 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:40.027930 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb6c3b8-015f-4eda-9734-1b2abd696a4d None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:40.067060 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cea3079-69a6-428a-9cd1-3c318f487d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:40.068299 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cea3079-69a6-428a-9cd1-3c318f487d69 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:40.068630 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cea3079-69a6-428a-9cd1-3c318f487d69 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:40.069049 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cea3079-69a6-428a-9cd1-3c318f487d69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:40.080200 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cea3079-69a6-428a-9cd1-3c318f487d69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:40.081914 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cea3079-69a6-428a-9cd1-3c318f487d69 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:40.102474 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-6cea3079-69a6-428a-9cd1-3c318f487d69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:40.102474 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:40.102474 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:40.102474 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:40.102474 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:40.102474 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:40.102474 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:40.102474 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:40.102474 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:40.102474 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:40.102474 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:40.102474 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:40.102474 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:20:40.102474 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:20:40.102474 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:20:40.103409 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:20:40.103409 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:40.103409 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:40.103409 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:40.103409 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:40.103409 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:40.103409 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:40.103409 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:40.103409 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:40.103409 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:40.103409 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:40.103409 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:40.103409 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:40.103409 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:40.103409 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:40.103409 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:40.103409 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:40.104282 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:20:40.104282 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:20:40.104282 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:40.104282 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:40.105869 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8882ef01-ffdb-4458-af04-aefbe9f8e2ff None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:40.109499 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 388/762] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:20:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:40.109705 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cea3079-69a6-428a-9cd1-3c318f487d69 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:40.152398 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62310af8-3ffc-40c8-ba62-02ef8d449080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:40.153355 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62310af8-3ffc-40c8-ba62-02ef8d449080 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:40.154217 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62310af8-3ffc-40c8-ba62-02ef8d449080 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:40.154688 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62310af8-3ffc-40c8-ba62-02ef8d449080 None admin] PATH_INFO: `/v3/services` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:40.155584 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62310af8-3ffc-40c8-ba62-02ef8d449080 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:40.156442 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62310af8-3ffc-40c8-ba62-02ef8d449080 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:40.158665 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62310af8-3ffc-40c8-ba62-02ef8d449080 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:40.173380 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 375/763] 10.222.0.85 () {56 vars in 1124 bytes} [Tue Jun 7 14:20:39 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:40.173380 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62310af8-3ffc-40c8-ba62-02ef8d449080 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:40.200408 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:40.234774 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cea3079-69a6-428a-9cd1-3c318f487d69 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:40.274496 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:40.275864 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281b44d8-afd8-4cf1-969b-749a1fa12277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:40.276618 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281b44d8-afd8-4cf1-969b-749a1fa12277 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:40.276925 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281b44d8-afd8-4cf1-969b-749a1fa12277 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:40.277179 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281b44d8-afd8-4cf1-969b-749a1fa12277 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:40.280336 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62310af8-3ffc-40c8-ba62-02ef8d449080 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:40.284360 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281b44d8-afd8-4cf1-969b-749a1fa12277 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:40.285855 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281b44d8-afd8-4cf1-969b-749a1fa12277 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:40.300609 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-281b44d8-afd8-4cf1-969b-749a1fa12277 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:20:40.300609 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:40.300609 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:40.300609 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:40.300609 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:40.300609 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:40.300609 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:40.300609 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:40.300609 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:40.300609 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:40.300609 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:40.300609 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:40.300609 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 14:20:40.300609 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 14:20:40.300609 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 14:20:40.301556 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 14:20:40.301556 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:40.301556 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:40.301556 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:40.301556 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:40.301556 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:40.301556 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:40.301556 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:40.301556 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:40.301556 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:40.301556 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:40.301556 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:40.301556 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:40.301556 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:40.301556 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:40.301556 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:40.301556 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:40.302441 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 14:20:40.302441 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 14:20:40.302441 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 14:20:40.302441 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 14:20:40.302441 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 14:20:40.302441 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 14:20:40.302441 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:20:40.302441 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:40.302441 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 389/764] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jun 7 14:20:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:40.303384 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cae723-e193-4f4a-a82a-ea31e14ec80c None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:40.303560 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cae723-e193-4f4a-a82a-ea31e14ec80c None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:40.303824 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cae723-e193-4f4a-a82a-ea31e14ec80c None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:40.308415 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 390/765] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:20:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:20:40.308415 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cae723-e193-4f4a-a82a-ea31e14ec80c None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:40.322629 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e3d935-7dc3-4b5e-89f2-a2051db15370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:40.323368 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e3d935-7dc3-4b5e-89f2-a2051db15370 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:40.323659 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e3d935-7dc3-4b5e-89f2-a2051db15370 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:40.323996 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e3d935-7dc3-4b5e-89f2-a2051db15370 None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:40.324784 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e3d935-7dc3-4b5e-89f2-a2051db15370 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:40.325153 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e3d935-7dc3-4b5e-89f2-a2051db15370 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:40.326558 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e3d935-7dc3-4b5e-89f2-a2051db15370 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:40.343098 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 376/766] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jun 7 14:20:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:40.343098 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b071061-1324-4ed2-8f7d-7ee3f957cf3a None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:40.343098 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b071061-1324-4ed2-8f7d-7ee3f957cf3a None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:40.343098 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b071061-1324-4ed2-8f7d-7ee3f957cf3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:40.357856 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-1b071061-1324-4ed2-8f7d-7ee3f957cf3a None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:20:40.359249 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-1b071061-1324-4ed2-8f7d-7ee3f957cf3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:20:40.429393 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cae723-e193-4f4a-a82a-ea31e14ec80c None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:40.429897 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 377/767] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:20:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:20:40.440796 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b071061-1324-4ed2-8f7d-7ee3f957cf3a None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:40.474498 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:40.479907 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6c5e75-bd96-41ad-bfd9-4aa466baa145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:40.480725 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6c5e75-bd96-41ad-bfd9-4aa466baa145 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:40.481076 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6c5e75-bd96-41ad-bfd9-4aa466baa145 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:40.481423 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6c5e75-bd96-41ad-bfd9-4aa466baa145 None admin] PATH_INFO: `/v3/services` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:40.492599 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6c5e75-bd96-41ad-bfd9-4aa466baa145 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:40.492599 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6c5e75-bd96-41ad-bfd9-4aa466baa145 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:40.492599 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6c5e75-bd96-41ad-bfd9-4aa466baa145 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:40.492599 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 391/768] 10.222.0.85 () {56 vars in 1124 bytes} [Tue Jun 7 14:20:40 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:40.492599 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6c5e75-bd96-41ad-bfd9-4aa466baa145 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:40.528627 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:40.547068 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b071061-1324-4ed2-8f7d-7ee3f957cf3a None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:40.586047 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1983fdf7-ee50-4c6f-b62f-8841aa261648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:40.586712 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1983fdf7-ee50-4c6f-b62f-8841aa261648 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:40.586997 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1983fdf7-ee50-4c6f-b62f-8841aa261648 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:40.587272 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1983fdf7-ee50-4c6f-b62f-8841aa261648 None admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:40.588064 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1983fdf7-ee50-4c6f-b62f-8841aa261648 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:40.588397 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1983fdf7-ee50-4c6f-b62f-8841aa261648 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:40.589824 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1983fdf7-ee50-4c6f-b62f-8841aa261648 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:40.596888 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6c5e75-bd96-41ad-bfd9-4aa466baa145 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:40.604821 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 378/769] 10.222.0.85 () {56 vars in 1148 bytes} [Tue Jun 7 14:20:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:40.605786 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1983fdf7-ee50-4c6f-b62f-8841aa261648 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:40.636235 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b8222c-ac0a-431c-8fe2-ea90dd74a224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:40.637365 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b8222c-ac0a-431c-8fe2-ea90dd74a224 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:40.637837 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b8222c-ac0a-431c-8fe2-ea90dd74a224 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:40.638289 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b8222c-ac0a-431c-8fe2-ea90dd74a224 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:40.649158 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b8222c-ac0a-431c-8fe2-ea90dd74a224 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:40.650862 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b8222c-ac0a-431c-8fe2-ea90dd74a224 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:40.666483 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-c5b8222c-ac0a-431c-8fe2-ea90dd74a224 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:20:40.666483 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:40.666483 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:40.666483 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:40.666483 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:40.666483 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:40.666483 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:40.666483 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:40.666483 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:40.666483 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:40.666483 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:40.666483 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:40.666483 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 14:20:40.666483 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 14:20:40.666483 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 14:20:40.667585 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 14:20:40.667585 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:40.667585 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:40.667585 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:40.667585 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:40.667585 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:40.667585 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:40.667585 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:40.667585 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:40.667585 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:40.667585 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:40.667585 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:40.667585 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:40.667585 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:40.667585 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:40.667585 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:40.667585 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:40.668708 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 14:20:40.668708 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 14:20:40.668708 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 14:20:40.668708 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 14:20:40.668708 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 14:20:40.668708 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 14:20:40.668708 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:20:40.668708 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:40.669770 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 392/770] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jun 7 14:20:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:40.670607 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b8222c-ac0a-431c-8fe2-ea90dd74a224 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:40.719987 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1983fdf7-ee50-4c6f-b62f-8841aa261648 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:40.736671 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:40.759733 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1411dc6-68d0-458f-8823-730a2343f200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:40.760563 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1411dc6-68d0-458f-8823-730a2343f200 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:40.760844 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1411dc6-68d0-458f-8823-730a2343f200 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:40.761120 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1411dc6-68d0-458f-8823-730a2343f200 None admin] PATH_INFO: `/v3/services/network` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:40.761781 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1411dc6-68d0-458f-8823-730a2343f200 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:40.763187 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1411dc6-68d0-458f-8823-730a2343f200 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:40.776339 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-b1411dc6-68d0-458f-8823-730a2343f200 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 07 14:20:40.776339 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:40.776339 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 14:20:40.776339 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 14:20:40.776339 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 14:20:40.776339 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 14:20:40.776339 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 14:20:40.776339 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:20:40.776339 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 07 14:20:40.776339 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:40.776339 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 14:20:40.776339 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:40.776339 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:40.776339 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:40.776339 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:40.776339 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:40.776339 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:40.777554 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:40.777554 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:40.777554 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:40.777554 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:40.777554 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:40.777554 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:40.777554 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 14:20:40.777554 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 14:20:40.777554 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 14:20:40.777554 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 14:20:40.777554 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:40.777554 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:40.777554 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:40.777554 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:40.777554 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:40.777554 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:40.778402 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:40.778402 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:40.778402 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:40.778402 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:40.778402 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:40.778402 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:40.778402 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:40.778402 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:40.778402 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:40.778402 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:40.778402 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 14:20:40.778402 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:20:40.778402 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 07 14:20:40.778402 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:40.778402 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 379/771] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jun 7 14:20:40 2022] GET /identity/v3/services/network => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:40.779205 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1411dc6-68d0-458f-8823-730a2343f200 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:40.792339 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:40.793630 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b8222c-ac0a-431c-8fe2-ea90dd74a224 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:40.828737 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ea0460-e028-4618-ab04-e89a3926e089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:40.829399 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ea0460-e028-4618-ab04-e89a3926e089 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:40.829682 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ea0460-e028-4618-ab04-e89a3926e089 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:40.829959 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ea0460-e028-4618-ab04-e89a3926e089 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:40.830757 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ea0460-e028-4618-ab04-e89a3926e089 None admin] RBAC: Adding query filter params (interface=public, service_id=ba22e3dd41b24716a238bf3f0e0127df, region_id=RegionOne) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:40.831066 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ea0460-e028-4618-ab04-e89a3926e089 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:40.832454 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ea0460-e028-4618-ab04-e89a3926e089 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:40.840634 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 393/772] 10.222.0.85 () {56 vars in 1251 bytes} [Tue Jun 7 14:20:40 2022] GET /identity/v3/endpoints?service_id=ba22e3dd41b24716a238bf3f0e0127df&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:40.841528 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ea0460-e028-4618-ab04-e89a3926e089 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:40.897525 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1411dc6-68d0-458f-8823-730a2343f200 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:40.933598 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4f8797-3772-46d8-a792-46f9fc0e7589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:40.934513 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f8797-3772-46d8-a792-46f9fc0e7589 None admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:40.934795 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f8797-3772-46d8-a792-46f9fc0e7589 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:40.935071 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f8797-3772-46d8-a792-46f9fc0e7589 None admin] PATH_INFO: `/v3/projects/724785eb74e44daf8736fd01fc0d1839/users/8e857db4902e42989d0d040bbc8517c8/roles/9baff2b18f2649f7b757ff2e1f63c853` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:40.937329 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4f8797-3772-46d8-a792-46f9fc0e7589 None admin] RBAC: Authorizing `identity:create_grant(project_id=724785eb74e44daf8736fd01fc0d1839, user_id=8e857db4902e42989d0d040bbc8517c8, role_id=9baff2b18f2649f7b757ff2e1f63c853)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:40.942628 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4f8797-3772-46d8-a792-46f9fc0e7589 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:40.950614 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ea0460-e028-4618-ab04-e89a3926e089 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:40.975827 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:40.989149 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348b0be3-4bb9-48d9-94ce-c6ea1196289d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:40.989912 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348b0be3-4bb9-48d9-94ce-c6ea1196289d None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:40.990091 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348b0be3-4bb9-48d9-94ce-c6ea1196289d None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:40.990363 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348b0be3-4bb9-48d9-94ce-c6ea1196289d None admin] PATH_INFO: `/v3/projects` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:40.991162 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348b0be3-4bb9-48d9-94ce-c6ea1196289d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:40.991469 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348b0be3-4bb9-48d9-94ce-c6ea1196289d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:40.992934 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348b0be3-4bb9-48d9-94ce-c6ea1196289d None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:41.004874 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 394/773] 10.222.0.85 () {56 vars in 1148 bytes} [Tue Jun 7 14:20:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:41.005464 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348b0be3-4bb9-48d9-94ce-c6ea1196289d None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:41.036880 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 380/774] 10.222.0.85 () {58 vars in 1439 bytes} [Tue Jun 7 14:20:40 2022] PUT /identity/v3/projects/724785eb74e44daf8736fd01fc0d1839/users/8e857db4902e42989d0d040bbc8517c8/roles/9baff2b18f2649f7b757ff2e1f63c853 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:20:41.044726 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4f8797-3772-46d8-a792-46f9fc0e7589 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:41.118897 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348b0be3-4bb9-48d9-94ce-c6ea1196289d None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:41.160022 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd27e4f-d32f-4007-a8f3-6107ac0fddcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:41.161063 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd27e4f-d32f-4007-a8f3-6107ac0fddcf None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:41.161549 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd27e4f-d32f-4007-a8f3-6107ac0fddcf None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:41.162011 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd27e4f-d32f-4007-a8f3-6107ac0fddcf None admin] PATH_INFO: `/v3/services` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:41.163248 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd27e4f-d32f-4007-a8f3-6107ac0fddcf None admin] RBAC: Adding query filter params (name=network) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:41.163810 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd27e4f-d32f-4007-a8f3-6107ac0fddcf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:41.165509 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd27e4f-d32f-4007-a8f3-6107ac0fddcf None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:41.171687 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:41.178856 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 395/775] 10.222.0.85 () {56 vars in 1112 bytes} [Tue Jun 7 14:20:41 2022] GET /identity/v3/services?name=network => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:41.178856 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd27e4f-d32f-4007-a8f3-6107ac0fddcf None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:41.213840 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4f8797-3772-46d8-a792-46f9fc0e7589 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:41.251268 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b15237-a136-431d-bdcd-5f50616fb362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:41.252286 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b15237-a136-431d-bdcd-5f50616fb362 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:41.252973 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b15237-a136-431d-bdcd-5f50616fb362 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:41.253458 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b15237-a136-431d-bdcd-5f50616fb362 None admin] PATH_INFO: `/v3/services` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:41.254363 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b15237-a136-431d-bdcd-5f50616fb362 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:41.256055 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b15237-a136-431d-bdcd-5f50616fb362 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:41.263626 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 381/776] 10.222.0.85 () {56 vars in 1087 bytes} [Tue Jun 7 14:20:41 2022] GET /identity/v3/services => generated 772 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:41.264812 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b15237-a136-431d-bdcd-5f50616fb362 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:41.294606 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:41.379047 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd27e4f-d32f-4007-a8f3-6107ac0fddcf None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:41.399839 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b15237-a136-431d-bdcd-5f50616fb362 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:41.419460 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbee8548-8c5b-4bb7-a25a-30465942699c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:41.420137 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbee8548-8c5b-4bb7-a25a-30465942699c None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:41.420438 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbee8548-8c5b-4bb7-a25a-30465942699c None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:41.420718 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbee8548-8c5b-4bb7-a25a-30465942699c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:41.421578 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbee8548-8c5b-4bb7-a25a-30465942699c None admin] RBAC: Adding query filter params (role.id=17516844e8b947b389d62f50718b852d, scope.project.id=724785eb74e44daf8736fd01fc0d1839, user.id=ad9d021b171244d796c320aa9f8450a9) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:41.421891 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbee8548-8c5b-4bb7-a25a-30465942699c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:41.423949 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbee8548-8c5b-4bb7-a25a-30465942699c None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:41.431117 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 396/777] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jun 7 14:20:41 2022] GET /identity/v3/role_assignments?user.id=ad9d021b171244d796c320aa9f8450a9&scope.project.id=724785eb74e44daf8736fd01fc0d1839&role.id=17516844e8b947b389d62f50718b852d => generated 259 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:41.432667 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ec7753-ba72-4fe4-9c8c-3b042cf91755 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:41.432944 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ec7753-ba72-4fe4-9c8c-3b042cf91755 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:41.433230 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ec7753-ba72-4fe4-9c8c-3b042cf91755 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:41.434819 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 397/778] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:20:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:20:41.441951 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1ca285-fde6-48dd-a589-f2bb064f7c99 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:41.442335 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1ca285-fde6-48dd-a589-f2bb064f7c99 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:41.442747 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1ca285-fde6-48dd-a589-f2bb064f7c99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:41.448514 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b70f5d-15ae-40ba-8f14-adfa0e3f28f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:41.449453 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b70f5d-15ae-40ba-8f14-adfa0e3f28f6 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:41.449910 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b70f5d-15ae-40ba-8f14-adfa0e3f28f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:41.450338 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b70f5d-15ae-40ba-8f14-adfa0e3f28f6 None admin] PATH_INFO: `/v3/services` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:41.451242 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b70f5d-15ae-40ba-8f14-adfa0e3f28f6 None admin] RBAC: Adding query filter params (type=network) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:41.451734 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b70f5d-15ae-40ba-8f14-adfa0e3f28f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:41.453756 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b70f5d-15ae-40ba-8f14-adfa0e3f28f6 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:41.455679 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:41.460873 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 382/779] 10.222.0.85 () {56 vars in 1112 bytes} [Tue Jun 7 14:20:41 2022] GET /identity/v3/services?type=network => generated 350 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:41.469790 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b70f5d-15ae-40ba-8f14-adfa0e3f28f6 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:41.470885 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-9d1ca285-fde6-48dd-a589-f2bb064f7c99 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:20:41.474593 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-9d1ca285-fde6-48dd-a589-f2bb064f7c99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:20:41.573923 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 398/780] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:20:41 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:20:41.598041 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1ca285-fde6-48dd-a589-f2bb064f7c99 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:41.613566 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b70f5d-15ae-40ba-8f14-adfa0e3f28f6 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:41.655359 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:41.659114 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc6607b-9ff3-4134-b3ad-ea72671ed66d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:41.659897 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc6607b-9ff3-4134-b3ad-ea72671ed66d None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:41.660255 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc6607b-9ff3-4134-b3ad-ea72671ed66d None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:41.660637 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc6607b-9ff3-4134-b3ad-ea72671ed66d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:41.661518 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc6607b-9ff3-4134-b3ad-ea72671ed66d None admin] RBAC: Adding query filter params (interface=public, service_id=197fb529b8ce447db9d3937983a4c36b, region_id=RegionOne) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:41.661912 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc6607b-9ff3-4134-b3ad-ea72671ed66d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:41.663406 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc6607b-9ff3-4134-b3ad-ea72671ed66d None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:41.674129 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 383/781] 10.222.0.85 () {56 vars in 1251 bytes} [Tue Jun 7 14:20:41 2022] GET /identity/v3/endpoints?service_id=197fb529b8ce447db9d3937983a4c36b&interface=public®ion_id=RegionOne => generated 194 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:41.680259 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc6607b-9ff3-4134-b3ad-ea72671ed66d None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:41.740171 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1ca285-fde6-48dd-a589-f2bb064f7c99 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:41.761469 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:41.790499 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c59ebdf-0202-4371-ad11-6befea45cd77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:41.792093 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c59ebdf-0202-4371-ad11-6befea45cd77 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:41.792877 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c59ebdf-0202-4371-ad11-6befea45cd77 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:41.793426 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c59ebdf-0202-4371-ad11-6befea45cd77 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:41.806983 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc6607b-9ff3-4134-b3ad-ea72671ed66d None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:41.829194 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c59ebdf-0202-4371-ad11-6befea45cd77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:20:41.837025 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-7c59ebdf-0202-4371-ad11-6befea45cd77 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:20:41.837025 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:41.837025 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:41.837025 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:41.837025 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:41.837025 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:41.837025 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:41.837025 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:41.837025 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:41.837025 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:41.837025 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:41.837025 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:41.837025 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:20:41.837025 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:20:41.837025 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:20:41.837025 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:20:41.837997 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:20:41.837997 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:20:41.837997 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:20:41.837997 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:20:41.837997 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:41.837997 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:41.837997 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:41.837997 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:41.837997 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:41.837997 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:41.837997 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:41.837997 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:41.837997 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:41.837997 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:41.837997 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:41.837997 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:41.838819 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:41.838819 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:41.838819 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:41.838819 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:41.838819 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:20:41.838819 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:20:41.838819 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:20:41.838819 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:20:41.838819 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:20:41.838819 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:20:41.838819 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:20:41.838819 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:41.838819 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 399/782] 10.222.0.85 () {56 vars in 1096 bytes} [Tue Jun 7 14:20:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:41.849442 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c59ebdf-0202-4371-ad11-6befea45cd77 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:41.869502 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d62c64-65dd-417a-8c19-54bc15e9f0d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:41.870814 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d62c64-65dd-417a-8c19-54bc15e9f0d9 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:41.871237 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d62c64-65dd-417a-8c19-54bc15e9f0d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:41.871670 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d62c64-65dd-417a-8c19-54bc15e9f0d9 None admin] PATH_INFO: `/v3/services` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:41.872575 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d62c64-65dd-417a-8c19-54bc15e9f0d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:41.874247 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d62c64-65dd-417a-8c19-54bc15e9f0d9 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:41.881222 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 384/783] 10.222.0.85 () {56 vars in 1087 bytes} [Tue Jun 7 14:20:41 2022] GET /identity/v3/services => generated 772 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:20:42.033399 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:42.045022 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c59ebdf-0202-4371-ad11-6befea45cd77 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:42.086680 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c669841d-eb73-4740-a227-5ae6d6d266d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:42.087610 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c669841d-eb73-4740-a227-5ae6d6d266d0 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:42.088054 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c669841d-eb73-4740-a227-5ae6d6d266d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:42.088514 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c669841d-eb73-4740-a227-5ae6d6d266d0 None admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:42.089460 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c669841d-eb73-4740-a227-5ae6d6d266d0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:42.089922 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c669841d-eb73-4740-a227-5ae6d6d266d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:42.091561 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c669841d-eb73-4740-a227-5ae6d6d266d0 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:42.102260 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 400/784] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jun 7 14:20:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:42.108236 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d62c64-65dd-417a-8c19-54bc15e9f0d9 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:42.245528 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d62c64-65dd-417a-8c19-54bc15e9f0d9 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:42.278391 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:42.282373 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1402e3d-67c1-4a60-b7bf-8bedccd604c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:42.283175 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1402e3d-67c1-4a60-b7bf-8bedccd604c8 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:42.283458 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1402e3d-67c1-4a60-b7bf-8bedccd604c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:42.283732 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1402e3d-67c1-4a60-b7bf-8bedccd604c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:42.296247 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1402e3d-67c1-4a60-b7bf-8bedccd604c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:42.297738 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1402e3d-67c1-4a60-b7bf-8bedccd604c8 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:42.316746 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-f1402e3d-67c1-4a60-b7bf-8bedccd604c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:42.316746 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:42.316746 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:42.316746 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:42.316746 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:42.316746 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:42.316746 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:42.316746 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:42.316746 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:42.316746 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:42.316746 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:42.316746 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:42.316746 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:20:42.316746 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:20:42.316746 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:20:42.317703 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:20:42.317703 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:42.317703 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:42.317703 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:42.317703 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:42.317703 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:42.317703 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:42.317703 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:42.317703 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:42.317703 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:42.317703 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:42.317703 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:42.317703 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:42.317703 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:42.317703 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:42.317703 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:42.317703 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:42.319085 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:20:42.319085 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:20:42.319085 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:42.319085 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:42.319085 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 385/785] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:20:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:42.324052 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c669841d-eb73-4740-a227-5ae6d6d266d0 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:42.470299 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c669841d-eb73-4740-a227-5ae6d6d266d0 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:42.515511 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7182707-bf2e-46c6-931d-4925b2932cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:42.516196 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7182707-bf2e-46c6-931d-4925b2932cee None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:42.516507 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7182707-bf2e-46c6-931d-4925b2932cee None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:42.516788 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7182707-bf2e-46c6-931d-4925b2932cee None admin] PATH_INFO: `/v3/domains` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:42.517521 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7182707-bf2e-46c6-931d-4925b2932cee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:42.517832 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7182707-bf2e-46c6-931d-4925b2932cee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:42.519210 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7182707-bf2e-46c6-931d-4925b2932cee None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:42.542834 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 401/786] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jun 7 14:20:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:42.549466 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1402e3d-67c1-4a60-b7bf-8bedccd604c8 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:42.715482 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1402e3d-67c1-4a60-b7bf-8bedccd604c8 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:42.752684 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0758cf3-cf3f-4b36-8ab6-d31e6e065aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:42.753401 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0758cf3-cf3f-4b36-8ab6-d31e6e065aed None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:42.753691 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0758cf3-cf3f-4b36-8ab6-d31e6e065aed None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:42.753970 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0758cf3-cf3f-4b36-8ab6-d31e6e065aed None admin] PATH_INFO: `/v3/users/swift` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:42.763150 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0758cf3-cf3f-4b36-8ab6-d31e6e065aed None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:42.764624 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0758cf3-cf3f-4b36-8ab6-d31e6e065aed None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:42.779356 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-e0758cf3-cf3f-4b36-8ab6-d31e6e065aed None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 07 14:20:42.779356 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:42.779356 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:42.779356 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:42.779356 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:42.779356 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:42.779356 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:42.779356 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:42.779356 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:42.779356 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:42.779356 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:42.779356 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:42.779356 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 14:20:42.779356 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 14:20:42.779356 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 14:20:42.779356 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 14:20:42.780316 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:42.780316 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:42.780316 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 14:20:42.780316 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:20:42.780316 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 14:20:42.780316 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:20:42.780316 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:42.780316 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:42.780316 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:42.780316 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:42.780316 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:42.780316 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:42.780316 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:42.780316 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:42.780316 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:42.780316 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:42.780316 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:42.780316 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:42.781462 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:42.781462 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:42.781462 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 14:20:42.781462 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 14:20:42.781462 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 14:20:42.781462 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 14:20:42.781462 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 14:20:42.781462 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 14:20:42.781462 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 07 14:20:42.781462 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:42.781462 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 386/787] 10.222.0.85 () {56 vars in 1096 bytes} [Tue Jun 7 14:20:42 2022] GET /identity/v3/users/swift => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:42.786545 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7182707-bf2e-46c6-931d-4925b2932cee None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:42.832839 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:42.950793 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7182707-bf2e-46c6-931d-4925b2932cee None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:43.018383 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebfa6c0-8ca9-4187-a446-0b299f5c4473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:43.019004 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebfa6c0-8ca9-4187-a446-0b299f5c4473 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:43.019291 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebfa6c0-8ca9-4187-a446-0b299f5c4473 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:43.019573 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebfa6c0-8ca9-4187-a446-0b299f5c4473 None admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:43.020455 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebfa6c0-8ca9-4187-a446-0b299f5c4473 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:43.020773 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebfa6c0-8ca9-4187-a446-0b299f5c4473 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:43.022776 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebfa6c0-8ca9-4187-a446-0b299f5c4473 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:43.056145 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 402/788] 10.222.0.85 () {56 vars in 1135 bytes} [Tue Jun 7 14:20:42 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:43.061319 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0758cf3-cf3f-4b36-8ab6-d31e6e065aed None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:43.126777 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:43.215236 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0758cf3-cf3f-4b36-8ab6-d31e6e065aed None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:43.282561 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf2725f-2a72-4384-9560-d3bc3d635de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:43.283280 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf2725f-2a72-4384-9560-d3bc3d635de1 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:43.283565 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf2725f-2a72-4384-9560-d3bc3d635de1 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:43.283841 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf2725f-2a72-4384-9560-d3bc3d635de1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:43.296304 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf2725f-2a72-4384-9560-d3bc3d635de1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:43.297786 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf2725f-2a72-4384-9560-d3bc3d635de1 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:43.317684 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-edf2725f-2a72-4384-9560-d3bc3d635de1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:43.317684 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:43.317684 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:43.317684 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:43.317684 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:43.317684 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:43.317684 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:43.317684 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:43.317684 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:43.317684 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:43.317684 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:43.317684 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:43.317684 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:20:43.317684 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:20:43.317684 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:20:43.319247 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:20:43.319247 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:43.319247 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:43.319247 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:43.319247 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:43.319247 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:43.319247 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:43.319247 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:43.319247 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:43.319247 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:43.319247 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:43.319247 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:43.319247 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:43.319247 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:43.319247 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:43.319247 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:43.319247 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:43.320138 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:20:43.320138 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:20:43.320138 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:43.320138 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:43.320138 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 387/789] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:20:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:43.325163 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebfa6c0-8ca9-4187-a446-0b299f5c4473 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:43.464261 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:43.488141 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebfa6c0-8ca9-4187-a446-0b299f5c4473 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:43.534579 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8f3e72-6236-47ca-8477-b603bbd710c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:43.535178 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8f3e72-6236-47ca-8477-b603bbd710c5 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:43.535465 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8f3e72-6236-47ca-8477-b603bbd710c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:43.535749 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8f3e72-6236-47ca-8477-b603bbd710c5 None admin] PATH_INFO: `/v3/domains` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:43.536512 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8f3e72-6236-47ca-8477-b603bbd710c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:43.536826 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8f3e72-6236-47ca-8477-b603bbd710c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:43.538758 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8f3e72-6236-47ca-8477-b603bbd710c5 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:43.572035 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 403/790] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jun 7 14:20:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:43.577480 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf2725f-2a72-4384-9560-d3bc3d635de1 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:43.703114 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:43.713424 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf2725f-2a72-4384-9560-d3bc3d635de1 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:43.762339 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde5edcd-16bc-450e-8973-2a7a81eab62c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:43.763196 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde5edcd-16bc-450e-8973-2a7a81eab62c None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:43.763483 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde5edcd-16bc-450e-8973-2a7a81eab62c None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:43.763869 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde5edcd-16bc-450e-8973-2a7a81eab62c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:43.774228 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde5edcd-16bc-450e-8973-2a7a81eab62c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:43.775679 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde5edcd-16bc-450e-8973-2a7a81eab62c None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:43.801067 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-fde5edcd-16bc-450e-8973-2a7a81eab62c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:20:43.801067 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:43.801067 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:43.801067 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:43.801067 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:43.801067 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:43.801067 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:43.801067 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:43.801067 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:43.801067 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:43.801067 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:43.801067 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:43.801067 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 14:20:43.801067 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 14:20:43.801067 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 14:20:43.802179 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 14:20:43.802179 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:43.802179 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:43.802179 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:43.802179 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:43.802179 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:43.802179 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:43.802179 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:43.802179 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:43.802179 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:43.802179 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:43.802179 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:43.802179 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:43.802179 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:43.802179 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:43.802179 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:43.802179 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:43.803367 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 14:20:43.803367 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 14:20:43.803367 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 14:20:43.803367 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 14:20:43.803367 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 14:20:43.803367 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 14:20:43.803367 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:20:43.803367 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:43.803367 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 388/791] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jun 7 14:20:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:43.808466 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8f3e72-6236-47ca-8477-b603bbd710c5 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:43.812404 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c66ee54-b0a3-494e-a18d-8e4d63d3b06d None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:43.812681 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c66ee54-b0a3-494e-a18d-8e4d63d3b06d None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:43.812963 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c66ee54-b0a3-494e-a18d-8e4d63d3b06d None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:43.814679 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 389/792] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:20:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 14:20:43.820680 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3df2e3c-21e6-45da-a19d-299d372cb9bb None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:43.820967 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3df2e3c-21e6-45da-a19d-299d372cb9bb None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:43.821304 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3df2e3c-21e6-45da-a19d-299d372cb9bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:43.840610 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-d3df2e3c-21e6-45da-a19d-299d372cb9bb None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:20:43.841878 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-d3df2e3c-21e6-45da-a19d-299d372cb9bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:20:43.914521 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 390/793] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:20:43 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 14:20:43.939134 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3df2e3c-21e6-45da-a19d-299d372cb9bb None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:44.024518 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:44.039536 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8f3e72-6236-47ca-8477-b603bbd710c5 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:44.060490 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3df2e3c-21e6-45da-a19d-299d372cb9bb None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:44.113576 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a351cdfe-40d4-43f1-b08b-f95973e93a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:44.114250 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a351cdfe-40d4-43f1-b08b-f95973e93a57 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:44.114534 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a351cdfe-40d4-43f1-b08b-f95973e93a57 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:44.114869 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a351cdfe-40d4-43f1-b08b-f95973e93a57 None admin] PATH_INFO: `/v3/projects` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:44.115747 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a351cdfe-40d4-43f1-b08b-f95973e93a57 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:44.116057 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a351cdfe-40d4-43f1-b08b-f95973e93a57 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:44.117524 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a351cdfe-40d4-43f1-b08b-f95973e93a57 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:44.135038 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6189e5c9-b778-46df-9cb5-043cf631a45b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:44.135709 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:44.135821 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6189e5c9-b778-46df-9cb5-043cf631a45b None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:44.136109 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6189e5c9-b778-46df-9cb5-043cf631a45b None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:44.136407 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6189e5c9-b778-46df-9cb5-043cf631a45b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:44.148550 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 404/794] 10.222.0.85 () {56 vars in 1148 bytes} [Tue Jun 7 14:20:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:44.154517 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a351cdfe-40d4-43f1-b08b-f95973e93a57 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:44.156156 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6189e5c9-b778-46df-9cb5-043cf631a45b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:20:44.164543 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-6189e5c9-b778-46df-9cb5-043cf631a45b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:20:44.164543 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:44.164543 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:44.164543 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:44.164543 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:44.164543 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:44.164543 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:44.164543 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:44.164543 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:44.164543 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:44.164543 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:44.164543 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:44.164543 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:20:44.164543 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:20:44.164543 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:20:44.164543 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:20:44.165509 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:20:44.165509 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:20:44.165509 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:20:44.165509 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:20:44.165509 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:44.165509 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:44.165509 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:44.165509 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:44.165509 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:44.165509 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:44.165509 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:44.165509 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:44.165509 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:44.165509 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:44.165509 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:44.165509 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:44.166907 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:44.166907 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:44.166907 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:44.166907 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:44.166907 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:20:44.166907 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:20:44.166907 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:20:44.166907 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:20:44.166907 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:20:44.166907 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:20:44.166907 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:20:44.166907 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:44.166907 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 391/795] 10.222.0.85 () {56 vars in 1096 bytes} [Tue Jun 7 14:20:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:44.171915 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6189e5c9-b778-46df-9cb5-043cf631a45b None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:44.306404 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a351cdfe-40d4-43f1-b08b-f95973e93a57 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:44.325492 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6189e5c9-b778-46df-9cb5-043cf631a45b None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:44.345008 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30317f0-0ab0-457a-8f3c-3297aade1166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:44.346058 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30317f0-0ab0-457a-8f3c-3297aade1166 None admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:44.346244 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30317f0-0ab0-457a-8f3c-3297aade1166 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:44.346479 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30317f0-0ab0-457a-8f3c-3297aade1166 None admin] PATH_INFO: `/v3/projects/724785eb74e44daf8736fd01fc0d1839/users/35aa67025f66459da0c0fc3cbfd45430/roles/9baff2b18f2649f7b757ff2e1f63c853` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:44.348562 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30317f0-0ab0-457a-8f3c-3297aade1166 None admin] RBAC: Authorizing `identity:create_grant(project_id=724785eb74e44daf8736fd01fc0d1839, user_id=35aa67025f66459da0c0fc3cbfd45430, role_id=9baff2b18f2649f7b757ff2e1f63c853)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:44.350052 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30317f0-0ab0-457a-8f3c-3297aade1166 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:44.370336 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9544cfe5-292c-4618-a7f6-6d4a05fd251f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:44.371109 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9544cfe5-292c-4618-a7f6-6d4a05fd251f None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:44.371486 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9544cfe5-292c-4618-a7f6-6d4a05fd251f None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:44.371835 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9544cfe5-292c-4618-a7f6-6d4a05fd251f None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:44.372701 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9544cfe5-292c-4618-a7f6-6d4a05fd251f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:44.373084 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9544cfe5-292c-4618-a7f6-6d4a05fd251f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:44.374562 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9544cfe5-292c-4618-a7f6-6d4a05fd251f None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:44.394653 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 392/796] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jun 7 14:20:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:20:44.394653 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb2dd39-7cef-4934-ab4d-c1d04fee8e6c None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:44.394653 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb2dd39-7cef-4934-ab4d-c1d04fee8e6c None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:44.394653 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb2dd39-7cef-4934-ab4d-c1d04fee8e6c None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:44.396474 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 393/797] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:20:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:20:44.397754 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb2dd39-7cef-4934-ab4d-c1d04fee8e6c None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:44.400826 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:44.420733 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 405/798] 10.222.0.85 () {58 vars in 1439 bytes} [Tue Jun 7 14:20:44 2022] PUT /identity/v3/projects/724785eb74e44daf8736fd01fc0d1839/users/35aa67025f66459da0c0fc3cbfd45430/roles/9baff2b18f2649f7b757ff2e1f63c853 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:20:44.429368 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c9b6c2-048c-4386-926f-544dd71f46bf None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:44.430090 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c9b6c2-048c-4386-926f-544dd71f46bf None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:44.430581 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c9b6c2-048c-4386-926f-544dd71f46bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:44.466463 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-63c9b6c2-048c-4386-926f-544dd71f46bf None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:20:44.468062 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-63c9b6c2-048c-4386-926f-544dd71f46bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:20:44.530257 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb2dd39-7cef-4934-ab4d-c1d04fee8e6c None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:44.562832 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:44.572214 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fa6875-f1f1-4ea5-9666-62691efdee25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:44.573549 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa6875-f1f1-4ea5-9666-62691efdee25 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:44.574045 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa6875-f1f1-4ea5-9666-62691efdee25 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:44.574527 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa6875-f1f1-4ea5-9666-62691efdee25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:44.587809 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fa6875-f1f1-4ea5-9666-62691efdee25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:44.591180 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fa6875-f1f1-4ea5-9666-62691efdee25 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:44.600419 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 406/799] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:20:44 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:20:44.614268 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea87d04-be1d-48f2-b344-53b491548c49 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:44.614610 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea87d04-be1d-48f2-b344-53b491548c49 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:44.614838 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea87d04-be1d-48f2-b344-53b491548c49 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:44.617178 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 407/800] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:20:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 14:20:44.630703 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea87d04-be1d-48f2-b344-53b491548c49 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:44.637816 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-28fa6875-f1f1-4ea5-9666-62691efdee25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:44.637816 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:44.637816 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:44.637816 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:44.637816 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:44.637816 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:44.637816 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:44.637816 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:44.637816 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:44.637816 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:44.637816 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:44.637816 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:44.637816 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:20:44.637816 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:20:44.637816 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:20:44.638980 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:20:44.638980 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:44.638980 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:44.638980 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:44.638980 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:44.638980 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:44.638980 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:44.638980 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:44.638980 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:44.638980 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:44.638980 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:44.638980 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:44.638980 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:44.638980 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:44.638980 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:44.638980 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:44.638980 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:44.639972 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:20:44.639972 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:20:44.639972 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:44.639972 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:44.640404 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 394/801] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:20:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:44.641448 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbca799e-2bc1-4340-b53f-022a8d012479 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:44.641737 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbca799e-2bc1-4340-b53f-022a8d012479 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:44.642018 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbca799e-2bc1-4340-b53f-022a8d012479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:44.660586 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-dbca799e-2bc1-4340-b53f-022a8d012479 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:20:44.661841 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-dbca799e-2bc1-4340-b53f-022a8d012479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:20:44.726777 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:44.754358 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 395/802] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:20:44 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:20:44.756455 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbca799e-2bc1-4340-b53f-022a8d012479 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:44.775634 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea87d04-be1d-48f2-b344-53b491548c49 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:44.821568 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c92f04-4475-4a8d-ae95-93d70fc411ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:44.822912 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c92f04-4475-4a8d-ae95-93d70fc411ac None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:44.823512 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c92f04-4475-4a8d-ae95-93d70fc411ac None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:44.823858 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c92f04-4475-4a8d-ae95-93d70fc411ac None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:44.824655 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c92f04-4475-4a8d-ae95-93d70fc411ac None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:44.826184 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c92f04-4475-4a8d-ae95-93d70fc411ac None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:44.840287 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-67c92f04-4475-4a8d-ae95-93d70fc411ac None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 07 14:20:44.840287 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:44.840287 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 14:20:44.840287 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 14:20:44.840287 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 14:20:44.840287 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 14:20:44.840287 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 14:20:44.840287 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:20:44.840287 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 07 14:20:44.840287 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:44.840287 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 14:20:44.840287 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:44.840287 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:44.840287 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:44.840287 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:44.840287 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:44.841281 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:44.841281 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:44.841281 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:44.841281 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:44.841281 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:44.841281 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:44.841281 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:44.841281 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 14:20:44.841281 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 14:20:44.841281 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 14:20:44.841281 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 14:20:44.841281 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:44.841281 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:44.841281 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:44.841281 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:44.841281 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:44.842106 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:44.842106 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:44.842106 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:44.842106 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:44.842106 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:44.842106 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:44.842106 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:44.842106 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:44.842106 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:44.842106 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:44.842106 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:44.842106 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 14:20:44.842106 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:20:44.842106 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 07 14:20:44.842106 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:44.842106 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 408/803] 10.222.0.85 () {56 vars in 1129 bytes} [Tue Jun 7 14:20:44 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:20:44.847380 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c92f04-4475-4a8d-ae95-93d70fc411ac None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:44.895982 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbca799e-2bc1-4340-b53f-022a8d012479 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:44.905014 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:44.941111 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23560795-79b6-4e07-958f-8be60c818016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:44.941920 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23560795-79b6-4e07-958f-8be60c818016 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:44.942282 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23560795-79b6-4e07-958f-8be60c818016 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:44.942635 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23560795-79b6-4e07-958f-8be60c818016 None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:44.943463 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23560795-79b6-4e07-958f-8be60c818016 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:44.943874 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23560795-79b6-4e07-958f-8be60c818016 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:44.945362 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23560795-79b6-4e07-958f-8be60c818016 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:44.958009 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c92f04-4475-4a8d-ae95-93d70fc411ac None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:44.960119 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 396/804] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jun 7 14:20:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:44.962405 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748e219a-8f33-4b23-bd63-bd641f37503e None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:44.962722 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748e219a-8f33-4b23-bd63-bd641f37503e None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:44.962990 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748e219a-8f33-4b23-bd63-bd641f37503e None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:44.964475 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 397/805] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:20:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:20:44.969219 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748e219a-8f33-4b23-bd63-bd641f37503e None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:45.001246 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ea4233-fc29-4d5e-a1a6-b52eaac9e844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:45.002046 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ea4233-fc29-4d5e-a1a6-b52eaac9e844 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:45.002394 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ea4233-fc29-4d5e-a1a6-b52eaac9e844 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:45.002705 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ea4233-fc29-4d5e-a1a6-b52eaac9e844 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:45.013737 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ea4233-fc29-4d5e-a1a6-b52eaac9e844 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:20:45.020344 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:45.022006 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-c8ea4233-fc29-4d5e-a1a6-b52eaac9e844 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:20:45.022006 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:45.022006 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:45.022006 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:45.022006 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:45.022006 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:45.022006 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:45.022006 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:45.022006 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:45.022006 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:45.022006 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:45.022006 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:45.022006 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:20:45.022006 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:20:45.022006 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:20:45.022006 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:20:45.022924 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:20:45.022924 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:20:45.022924 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:20:45.022924 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:20:45.022924 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:45.022924 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:45.022924 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:45.022924 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:45.022924 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:45.022924 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:45.022924 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:45.022924 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:45.022924 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:45.022924 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:45.022924 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:45.022924 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:45.023742 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:45.023742 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:45.023742 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:45.023742 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:45.023742 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:20:45.023742 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:20:45.023742 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:20:45.023742 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:20:45.023742 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:20:45.023742 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:20:45.023742 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:20:45.023742 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:45.023742 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 409/806] 10.222.0.85 () {56 vars in 1102 bytes} [Tue Jun 7 14:20:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:45.023742 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ea4233-fc29-4d5e-a1a6-b52eaac9e844 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:45.086382 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748e219a-8f33-4b23-bd63-bd641f37503e None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:45.124728 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ead5c4-f730-4598-83b1-b6b4ba80098d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:45.125320 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ead5c4-f730-4598-83b1-b6b4ba80098d None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:45.125615 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ead5c4-f730-4598-83b1-b6b4ba80098d None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:45.125899 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ead5c4-f730-4598-83b1-b6b4ba80098d None admin] PATH_INFO: `/v3/services` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:45.126620 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ead5c4-f730-4598-83b1-b6b4ba80098d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:45.126934 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ead5c4-f730-4598-83b1-b6b4ba80098d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:45.128308 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ead5c4-f730-4598-83b1-b6b4ba80098d None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:45.133395 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 398/807] 10.222.0.85 () {56 vars in 1124 bytes} [Tue Jun 7 14:20:44 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:45.134473 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d20ea52-c81d-43b5-a85e-701b419ead80 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:45.135080 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d20ea52-c81d-43b5-a85e-701b419ead80 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:45.135368 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d20ea52-c81d-43b5-a85e-701b419ead80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:45.147574 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ea4233-fc29-4d5e-a1a6-b52eaac9e844 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:45.153550 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-9d20ea52-c81d-43b5-a85e-701b419ead80 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:20:45.154735 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-9d20ea52-c81d-43b5-a85e-701b419ead80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:20:45.164138 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:45.187927 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab914d6b-543b-4827-8e69-88fb7d4e7f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:45.189012 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab914d6b-543b-4827-8e69-88fb7d4e7f08 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:45.189304 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab914d6b-543b-4827-8e69-88fb7d4e7f08 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:45.189581 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab914d6b-543b-4827-8e69-88fb7d4e7f08 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:45.199507 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab914d6b-543b-4827-8e69-88fb7d4e7f08 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:45.201673 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab914d6b-543b-4827-8e69-88fb7d4e7f08 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:45.224060 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-ab914d6b-543b-4827-8e69-88fb7d4e7f08 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 07 14:20:45.224060 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:45.224060 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:45.224060 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:45.224060 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:45.224060 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:45.224060 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:45.224060 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:45.224060 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:45.224060 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:45.224060 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:45.224060 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:45.224060 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 14:20:45.224060 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 14:20:45.224060 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 14:20:45.224060 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 14:20:45.225855 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:45.225855 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:45.225855 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 14:20:45.225855 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:20:45.225855 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 14:20:45.225855 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:20:45.225855 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:45.225855 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:45.225855 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:45.225855 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:45.225855 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:45.225855 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:45.225855 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:45.225855 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:45.225855 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:45.225855 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:45.225855 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:45.225855 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:45.226804 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:45.226804 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:45.226804 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 14:20:45.226804 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 14:20:45.226804 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 14:20:45.226804 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 14:20:45.226804 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 14:20:45.226804 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 14:20:45.226804 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 07 14:20:45.226804 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:45.228602 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 410/808] 10.222.0.85 () {56 vars in 1093 bytes} [Tue Jun 7 14:20:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:45.229425 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab914d6b-543b-4827-8e69-88fb7d4e7f08 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:45.248809 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 399/809] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:20:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:20:45.248809 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d20ea52-c81d-43b5-a85e-701b419ead80 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:45.297626 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:45.349767 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab914d6b-543b-4827-8e69-88fb7d4e7f08 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:45.367496 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d20ea52-c81d-43b5-a85e-701b419ead80 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:45.386270 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cda7aef-1295-4903-8810-20a106c4fe47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:45.386861 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cda7aef-1295-4903-8810-20a106c4fe47 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:45.387162 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cda7aef-1295-4903-8810-20a106c4fe47 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:45.387445 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cda7aef-1295-4903-8810-20a106c4fe47 None admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:45.388206 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cda7aef-1295-4903-8810-20a106c4fe47 None admin] RBAC: Adding query filter params (name=service) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:45.388552 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cda7aef-1295-4903-8810-20a106c4fe47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:45.389943 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cda7aef-1295-4903-8810-20a106c4fe47 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:45.399678 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 411/810] 10.222.0.85 () {56 vars in 1103 bytes} [Tue Jun 7 14:20:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:45.400358 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cda7aef-1295-4903-8810-20a106c4fe47 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:45.413405 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6574de5-ed9c-4311-9dc6-9bddc4545303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:45.413939 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6574de5-ed9c-4311-9dc6-9bddc4545303 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:45.414225 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6574de5-ed9c-4311-9dc6-9bddc4545303 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:45.414507 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6574de5-ed9c-4311-9dc6-9bddc4545303 None admin] PATH_INFO: `/v3/services` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:45.415353 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6574de5-ed9c-4311-9dc6-9bddc4545303 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:45.415525 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6574de5-ed9c-4311-9dc6-9bddc4545303 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:45.416966 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6574de5-ed9c-4311-9dc6-9bddc4545303 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:45.422499 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 400/811] 10.222.0.85 () {56 vars in 1124 bytes} [Tue Jun 7 14:20:45 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:45.423097 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6574de5-ed9c-4311-9dc6-9bddc4545303 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:45.451447 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:45.526417 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cda7aef-1295-4903-8810-20a106c4fe47 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:45.563169 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5968b479-f605-4407-853c-80b57cd0e9f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:45.563958 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5968b479-f605-4407-853c-80b57cd0e9f8 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:45.564118 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5968b479-f605-4407-853c-80b57cd0e9f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:45.564411 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5968b479-f605-4407-853c-80b57cd0e9f8 None admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:45.565236 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6574de5-ed9c-4311-9dc6-9bddc4545303 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:45.565337 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5968b479-f605-4407-853c-80b57cd0e9f8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:45.565564 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5968b479-f605-4407-853c-80b57cd0e9f8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:45.567002 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5968b479-f605-4407-853c-80b57cd0e9f8 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:45.567165 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:45.592414 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 412/812] 10.222.0.85 () {56 vars in 1133 bytes} [Tue Jun 7 14:20:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:45.592414 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5968b479-f605-4407-853c-80b57cd0e9f8 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:45.605434 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4493e81-20f9-4d10-867d-9afa899cc51c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:45.606234 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4493e81-20f9-4d10-867d-9afa899cc51c None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:45.606519 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4493e81-20f9-4d10-867d-9afa899cc51c None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:45.606795 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4493e81-20f9-4d10-867d-9afa899cc51c None admin] PATH_INFO: `/v3/services/network` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:45.607454 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4493e81-20f9-4d10-867d-9afa899cc51c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:45.608890 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4493e81-20f9-4d10-867d-9afa899cc51c None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:45.621228 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-b4493e81-20f9-4d10-867d-9afa899cc51c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 07 14:20:45.621228 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:45.621228 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 14:20:45.621228 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 14:20:45.621228 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 14:20:45.621228 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 14:20:45.621228 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 14:20:45.621228 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:20:45.621228 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 07 14:20:45.621228 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:45.621228 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 14:20:45.621228 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:45.621228 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:45.621228 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:45.621228 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:45.621228 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:45.621228 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:45.622252 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:45.622252 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:45.622252 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:45.622252 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:45.622252 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:45.622252 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:45.622252 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 14:20:45.622252 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 14:20:45.622252 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 14:20:45.622252 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 14:20:45.622252 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:45.622252 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:45.622252 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:45.622252 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:45.622252 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:45.622252 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:45.623107 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:45.623107 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:45.623107 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:45.623107 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:45.623107 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:45.623107 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:45.623107 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:45.623107 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:45.623107 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:45.623107 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:45.623107 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 14:20:45.623107 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:20:45.623107 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 07 14:20:45.623107 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:45.623107 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 401/813] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jun 7 14:20:45 2022] GET /identity/v3/services/network => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:45.623886 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4493e81-20f9-4d10-867d-9afa899cc51c None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:45.695189 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:45.703565 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5968b479-f605-4407-853c-80b57cd0e9f8 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:45.738324 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadcfc1d-be4f-4d3f-a215-3891a4e11ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:45.739734 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadcfc1d-be4f-4d3f-a215-3891a4e11ab8 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:45.740300 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadcfc1d-be4f-4d3f-a215-3891a4e11ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:45.741044 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadcfc1d-be4f-4d3f-a215-3891a4e11ab8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:45.741249 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4493e81-20f9-4d10-867d-9afa899cc51c None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:45.754906 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadcfc1d-be4f-4d3f-a215-3891a4e11ab8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:45.756937 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadcfc1d-be4f-4d3f-a215-3891a4e11ab8 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:45.776930 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-dadcfc1d-be4f-4d3f-a215-3891a4e11ab8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:45.776930 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:45.776930 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:45.776930 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:45.776930 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:45.776930 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:45.776930 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:45.776930 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:45.776930 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:45.776930 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:45.776930 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:45.776930 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:45.776930 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:20:45.776930 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:20:45.776930 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:20:45.778092 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:20:45.778092 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:45.778092 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:45.778092 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:45.778092 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:45.778092 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:45.778092 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:45.778092 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:45.778092 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:45.778092 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:45.778092 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:45.778092 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:45.778092 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:45.778092 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:45.778092 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:45.778092 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:45.778092 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:45.779004 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:20:45.779004 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:20:45.779004 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:45.779004 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:45.779004 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 413/814] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:20:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:45.779463 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadcfc1d-be4f-4d3f-a215-3891a4e11ab8 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:45.787407 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e231cc9-ccd4-4a20-ad85-b63223d75c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:45.788089 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e231cc9-ccd4-4a20-ad85-b63223d75c26 None admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:45.788396 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e231cc9-ccd4-4a20-ad85-b63223d75c26 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:45.788733 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e231cc9-ccd4-4a20-ad85-b63223d75c26 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:45.789542 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e231cc9-ccd4-4a20-ad85-b63223d75c26 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:45.790906 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e231cc9-ccd4-4a20-ad85-b63223d75c26 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:45.888014 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadcfc1d-be4f-4d3f-a215-3891a4e11ab8 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:45.901847 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 402/815] 10.222.0.85 () {60 vars in 1144 bytes} [Tue Jun 7 14:20:45 2022] POST /identity/v3/endpoints => generated 346 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:20:45.902820 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e231cc9-ccd4-4a20-ad85-b63223d75c26 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:45.950876 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6d3287-55af-4f19-afe5-8985e0f0cac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:45.950876 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6d3287-55af-4f19-afe5-8985e0f0cac8 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:45.950876 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6d3287-55af-4f19-afe5-8985e0f0cac8 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:45.950876 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6d3287-55af-4f19-afe5-8985e0f0cac8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:45.957061 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:45.971897 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:46.041285 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6d3287-55af-4f19-afe5-8985e0f0cac8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:46.043126 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6d3287-55af-4f19-afe5-8985e0f0cac8 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:46.059614 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e231cc9-ccd4-4a20-ad85-b63223d75c26 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:46.117025 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973c9e72-b58c-429e-ba3c-2507f208fae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:46.117700 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973c9e72-b58c-429e-ba3c-2507f208fae2 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:46.117990 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973c9e72-b58c-429e-ba3c-2507f208fae2 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:46.118925 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973c9e72-b58c-429e-ba3c-2507f208fae2 None admin] PATH_INFO: `/v3/services` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:46.119260 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973c9e72-b58c-429e-ba3c-2507f208fae2 None admin] RBAC: Adding query filter params (name=network) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:46.119574 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973c9e72-b58c-429e-ba3c-2507f208fae2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:46.121259 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973c9e72-b58c-429e-ba3c-2507f208fae2 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:46.126611 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 403/816] 10.222.0.85 () {56 vars in 1112 bytes} [Tue Jun 7 14:20:45 2022] GET /identity/v3/services?name=network => generated 124 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:46.131449 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973c9e72-b58c-429e-ba3c-2507f208fae2 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:46.179335 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 414/817] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jun 7 14:20:45 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 14:20:46.180943 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6d3287-55af-4f19-afe5-8985e0f0cac8 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:46.237623 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:46.249342 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:46.267464 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973c9e72-b58c-429e-ba3c-2507f208fae2 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:46.303270 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6d3287-55af-4f19-afe5-8985e0f0cac8 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:46.306776 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73558bab-815e-4adc-90b0-4ef6be49f0a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:46.307463 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73558bab-815e-4adc-90b0-4ef6be49f0a8 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:46.307751 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73558bab-815e-4adc-90b0-4ef6be49f0a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:46.308028 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73558bab-815e-4adc-90b0-4ef6be49f0a8 None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:46.308797 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73558bab-815e-4adc-90b0-4ef6be49f0a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:46.309108 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73558bab-815e-4adc-90b0-4ef6be49f0a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:46.310512 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73558bab-815e-4adc-90b0-4ef6be49f0a8 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:46.325036 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 404/818] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jun 7 14:20:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:46.325970 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73558bab-815e-4adc-90b0-4ef6be49f0a8 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:46.344341 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b0d6d1-9ff8-4420-9d14-f9a87f5c1a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:46.345304 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b0d6d1-9ff8-4420-9d14-f9a87f5c1a94 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:46.345598 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b0d6d1-9ff8-4420-9d14-f9a87f5c1a94 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:46.345881 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b0d6d1-9ff8-4420-9d14-f9a87f5c1a94 None admin] PATH_INFO: `/v3/services` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:46.346605 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b0d6d1-9ff8-4420-9d14-f9a87f5c1a94 None admin] RBAC: Adding query filter params (type=network) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:46.346915 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b0d6d1-9ff8-4420-9d14-f9a87f5c1a94 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:46.348397 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b0d6d1-9ff8-4420-9d14-f9a87f5c1a94 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:46.354421 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 415/819] 10.222.0.85 () {56 vars in 1112 bytes} [Tue Jun 7 14:20:46 2022] GET /identity/v3/services?type=network => generated 350 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:46.355264 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b0d6d1-9ff8-4420-9d14-f9a87f5c1a94 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:46.442441 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73558bab-815e-4adc-90b0-4ef6be49f0a8 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:46.466427 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b0d6d1-9ff8-4420-9d14-f9a87f5c1a94 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:46.469164 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:46.484139 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aff363-0cf0-4153-975e-437fca6c3236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:46.485001 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aff363-0cf0-4153-975e-437fca6c3236 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:46.485289 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aff363-0cf0-4153-975e-437fca6c3236 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:46.488970 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aff363-0cf0-4153-975e-437fca6c3236 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:46.498330 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6aff363-0cf0-4153-975e-437fca6c3236 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:46.499806 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6aff363-0cf0-4153-975e-437fca6c3236 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:46.505927 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d2bd7c-ee48-4aee-a4a5-35f144a447a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:46.506779 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d2bd7c-ee48-4aee-a4a5-35f144a447a1 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:46.507071 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d2bd7c-ee48-4aee-a4a5-35f144a447a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:46.507355 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d2bd7c-ee48-4aee-a4a5-35f144a447a1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:46.508335 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:46.515983 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d2bd7c-ee48-4aee-a4a5-35f144a447a1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:46.517549 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d2bd7c-ee48-4aee-a4a5-35f144a447a1 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:46.522052 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-a6aff363-0cf0-4153-975e-437fca6c3236 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:46.522052 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:46.522052 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:46.522052 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:46.522052 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:46.522052 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:46.522052 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:46.522052 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:46.522052 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:46.522052 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:46.522052 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:46.522052 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:46.522052 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:20:46.522052 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:20:46.522052 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:20:46.523128 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:20:46.523128 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:46.523128 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:46.523128 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:46.523128 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:46.523128 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:46.523128 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:46.523128 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:46.523128 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:46.523128 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:46.523128 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:46.523128 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:46.523128 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:46.523128 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:46.523128 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:46.523128 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:46.523128 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:46.524181 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:20:46.524181 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:20:46.524181 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:46.524181 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:46.526048 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 405/820] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:20:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:46.527075 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aff363-0cf0-4153-975e-437fca6c3236 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:46.540703 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-57d2bd7c-ee48-4aee-a4a5-35f144a447a1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 14:20:46.540703 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:46.540703 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:46.540703 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:46.540703 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:46.540703 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:46.540703 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:46.540703 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:46.540703 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:46.540703 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:46.540703 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:46.540703 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:46.540703 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 14:20:46.540703 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 14:20:46.540703 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 14:20:46.540703 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 14:20:46.541922 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:46.541922 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:46.541922 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 14:20:46.541922 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:20:46.541922 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 14:20:46.541922 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:20:46.541922 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:46.541922 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:46.541922 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:46.541922 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:46.541922 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:46.541922 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:46.541922 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:46.541922 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:46.541922 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:46.541922 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:46.541922 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:46.541922 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:46.542881 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:46.542881 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:46.542881 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 14:20:46.542881 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 14:20:46.542881 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 14:20:46.542881 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 14:20:46.542881 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 14:20:46.542881 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 14:20:46.542881 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 14:20:46.542881 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:46.542881 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 416/821] 10.222.0.85 () {56 vars in 1117 bytes} [Tue Jun 7 14:20:46 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:46.543471 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d2bd7c-ee48-4aee-a4a5-35f144a447a1 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:46.661041 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aff363-0cf0-4153-975e-437fca6c3236 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:46.696142 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d2bd7c-ee48-4aee-a4a5-35f144a447a1 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:46.707430 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033b10df-25fd-4b38-9863-8346bc21ca70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:46.708496 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033b10df-25fd-4b38-9863-8346bc21ca70 None admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:46.708983 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033b10df-25fd-4b38-9863-8346bc21ca70 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:46.709411 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033b10df-25fd-4b38-9863-8346bc21ca70 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:46.710371 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033b10df-25fd-4b38-9863-8346bc21ca70 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:46.712031 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033b10df-25fd-4b38-9863-8346bc21ca70 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:46.733614 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f209d2-1884-451c-941d-eadafbefaaaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:46.734242 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f209d2-1884-451c-941d-eadafbefaaaa None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:46.734527 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f209d2-1884-451c-941d-eadafbefaaaa None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:46.791268 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f209d2-1884-451c-941d-eadafbefaaaa None admin] PATH_INFO: `/v3/domains` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:46.791268 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f209d2-1884-451c-941d-eadafbefaaaa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:46.791268 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f209d2-1884-451c-941d-eadafbefaaaa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:46.791268 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f209d2-1884-451c-941d-eadafbefaaaa None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:46.791268 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 417/822] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jun 7 14:20:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:46.791268 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f209d2-1884-451c-941d-eadafbefaaaa None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:46.791711 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:46.848254 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 406/823] 10.222.0.85 () {60 vars in 1144 bytes} [Tue Jun 7 14:20:46 2022] POST /identity/v3/endpoints => generated 327 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:20:46.853355 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033b10df-25fd-4b38-9863-8346bc21ca70 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:46.871538 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f209d2-1884-451c-941d-eadafbefaaaa None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:46.933384 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fab1fb3-8d9e-46ce-8550-9ff6d58a6840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:46.934141 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fab1fb3-8d9e-46ce-8550-9ff6d58a6840 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:46.934429 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fab1fb3-8d9e-46ce-8550-9ff6d58a6840 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:46.934710 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fab1fb3-8d9e-46ce-8550-9ff6d58a6840 None admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:46.936093 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fab1fb3-8d9e-46ce-8550-9ff6d58a6840 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:46.936424 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fab1fb3-8d9e-46ce-8550-9ff6d58a6840 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:46.938148 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fab1fb3-8d9e-46ce-8550-9ff6d58a6840 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:46.951379 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:46.973919 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 418/824] 10.222.0.85 () {56 vars in 1149 bytes} [Tue Jun 7 14:20:46 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:46.980771 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fab1fb3-8d9e-46ce-8550-9ff6d58a6840 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:47.050106 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033b10df-25fd-4b38-9863-8346bc21ca70 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:47.098513 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf22d42-e97b-432a-82d8-bc09e634020c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:47.099513 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf22d42-e97b-432a-82d8-bc09e634020c None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:47.099802 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf22d42-e97b-432a-82d8-bc09e634020c None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:47.100662 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf22d42-e97b-432a-82d8-bc09e634020c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:47.109951 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf22d42-e97b-432a-82d8-bc09e634020c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:47.111750 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf22d42-e97b-432a-82d8-bc09e634020c None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:47.133998 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:47.138327 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-ddf22d42-e97b-432a-82d8-bc09e634020c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:20:47.138327 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:47.138327 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:47.138327 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:47.138327 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:47.138327 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:47.138327 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:47.138327 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:47.138327 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:47.138327 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:47.138327 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:47.138327 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:47.138327 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 14:20:47.138327 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 14:20:47.138327 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 14:20:47.146583 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 14:20:47.146583 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:47.146583 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:47.146583 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:47.146583 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:47.146583 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:47.146583 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:47.146583 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:47.146583 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:47.146583 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:47.146583 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:47.146583 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:47.146583 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:47.146583 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:47.146583 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:47.146583 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:47.146583 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:47.147531 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 14:20:47.147531 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 14:20:47.147531 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 14:20:47.147531 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 14:20:47.147531 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 14:20:47.147531 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 14:20:47.147531 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:20:47.147531 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:47.147531 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 407/825] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jun 7 14:20:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:47.147531 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f74d824-5922-45e9-b52e-1e656268e3b9 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:47.147531 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f74d824-5922-45e9-b52e-1e656268e3b9 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:47.148124 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f74d824-5922-45e9-b52e-1e656268e3b9 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:47.148124 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 408/826] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:20:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:20:47.153220 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775a50c0-7784-4473-a1bd-4c104aa3410e None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:47.153590 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775a50c0-7784-4473-a1bd-4c104aa3410e None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:47.154351 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775a50c0-7784-4473-a1bd-4c104aa3410e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:47.168668 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fab1fb3-8d9e-46ce-8550-9ff6d58a6840 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:47.174467 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-775a50c0-7784-4473-a1bd-4c104aa3410e None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:20:47.177785 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-775a50c0-7784-4473-a1bd-4c104aa3410e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:20:47.222729 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb73809e-11d8-47cc-b692-73d79ee901c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:47.223569 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb73809e-11d8-47cc-b692-73d79ee901c6 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:47.223852 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb73809e-11d8-47cc-b692-73d79ee901c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:47.224126 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb73809e-11d8-47cc-b692-73d79ee901c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:47.240468 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb73809e-11d8-47cc-b692-73d79ee901c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:47.241925 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb73809e-11d8-47cc-b692-73d79ee901c6 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:47.269722 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-fb73809e-11d8-47cc-b692-73d79ee901c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:47.269722 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:47.269722 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:47.269722 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:47.269722 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:47.269722 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:47.269722 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:47.269722 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:47.269722 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:47.269722 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:47.269722 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:47.269722 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:47.269722 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:20:47.269722 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:20:47.269722 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:20:47.271302 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:20:47.271302 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:47.271302 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:47.271302 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:47.271302 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:47.271302 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:47.271302 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:47.271302 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:47.271302 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:47.271302 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:47.271302 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:47.271302 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:47.271302 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:47.271302 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:47.271302 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:47.271302 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:47.271302 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:47.272235 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:20:47.272235 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:20:47.272235 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:47.272235 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:47.274591 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 419/827] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:20:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:47.275176 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb73809e-11d8-47cc-b692-73d79ee901c6 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:47.287073 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 409/828] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:20:47 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:20:47.288498 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775a50c0-7784-4473-a1bd-4c104aa3410e None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:47.357879 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:47.418577 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb73809e-11d8-47cc-b692-73d79ee901c6 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:47.441953 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775a50c0-7784-4473-a1bd-4c104aa3410e None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:47.461979 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:47.478817 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7edc956-04b4-4a5f-ae73-5f4019d8e323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:47.479495 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7edc956-04b4-4a5f-ae73-5f4019d8e323 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:47.479813 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7edc956-04b4-4a5f-ae73-5f4019d8e323 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:47.480063 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7edc956-04b4-4a5f-ae73-5f4019d8e323 None admin] PATH_INFO: `/v3/projects` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:47.480908 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7edc956-04b4-4a5f-ae73-5f4019d8e323 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:47.481141 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125d2c75-377c-4fa2-b957-cbe6e9be0147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:47.481259 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7edc956-04b4-4a5f-ae73-5f4019d8e323 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:47.481795 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d2c75-377c-4fa2-b957-cbe6e9be0147 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:47.482084 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d2c75-377c-4fa2-b957-cbe6e9be0147 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:47.482368 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d2c75-377c-4fa2-b957-cbe6e9be0147 None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:47.482641 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7edc956-04b4-4a5f-ae73-5f4019d8e323 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:47.483101 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125d2c75-377c-4fa2-b957-cbe6e9be0147 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:47.483412 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125d2c75-377c-4fa2-b957-cbe6e9be0147 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:47.484814 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125d2c75-377c-4fa2-b957-cbe6e9be0147 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:47.502991 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 410/829] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jun 7 14:20:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:47.502991 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125d2c75-377c-4fa2-b957-cbe6e9be0147 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:47.509768 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 420/830] 10.222.0.85 () {56 vars in 1148 bytes} [Tue Jun 7 14:20:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:47.509768 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7edc956-04b4-4a5f-ae73-5f4019d8e323 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:47.639294 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125d2c75-377c-4fa2-b957-cbe6e9be0147 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:47.642056 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:47.646456 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7edc956-04b4-4a5f-ae73-5f4019d8e323 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:47.680445 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8998c96d-a8b5-4c06-8111-97d402c78722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:47.681307 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8998c96d-a8b5-4c06-8111-97d402c78722 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:47.681676 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8998c96d-a8b5-4c06-8111-97d402c78722 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:47.682355 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8998c96d-a8b5-4c06-8111-97d402c78722 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:47.687621 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc158ba-c1dc-4354-958d-9cf4528cd931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:47.689795 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc158ba-c1dc-4354-958d-9cf4528cd931 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:47.689795 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc158ba-c1dc-4354-958d-9cf4528cd931 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:47.689795 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc158ba-c1dc-4354-958d-9cf4528cd931 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:47.697295 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8998c96d-a8b5-4c06-8111-97d402c78722 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:20:47.698415 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc158ba-c1dc-4354-958d-9cf4528cd931 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:47.699887 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc158ba-c1dc-4354-958d-9cf4528cd931 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:47.707535 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-8998c96d-a8b5-4c06-8111-97d402c78722 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:20:47.707535 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:47.707535 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:47.707535 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:47.707535 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:47.707535 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:47.707535 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:47.707535 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:47.707535 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:47.707535 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:47.707535 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:47.707535 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:47.707535 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:20:47.707535 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:20:47.707535 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:20:47.707535 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:20:47.712775 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:20:47.712775 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:20:47.712775 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:20:47.712775 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:20:47.712775 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:47.712775 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:47.712775 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:47.712775 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:47.712775 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:47.712775 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:47.712775 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:47.712775 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:47.712775 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:47.712775 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:47.712775 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:47.712775 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:47.715943 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:47.715943 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:47.715943 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:47.715943 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:47.715943 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:20:47.715943 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:20:47.715943 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:20:47.715943 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:20:47.715943 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:20:47.715943 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:20:47.715943 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:20:47.715943 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:47.715943 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 411/831] 10.222.0.85 () {56 vars in 1096 bytes} [Tue Jun 7 14:20:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:47.715943 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8998c96d-a8b5-4c06-8111-97d402c78722 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:47.725739 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-8fc158ba-c1dc-4354-958d-9cf4528cd931 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:20:47.725739 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:47.725739 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:47.725739 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:47.725739 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:47.725739 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:47.725739 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:47.725739 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:47.725739 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:47.725739 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:47.725739 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:47.725739 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:47.725739 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 14:20:47.725739 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 14:20:47.725739 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 14:20:47.728522 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 14:20:47.728522 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:47.728522 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:47.728522 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:47.728522 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:47.728522 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:47.728522 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:47.728522 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:47.728522 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:47.728522 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:47.728522 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:47.728522 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:47.728522 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:47.728522 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:47.728522 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:47.728522 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:47.728522 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:47.734115 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 14:20:47.734115 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 14:20:47.734115 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 14:20:47.734115 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 14:20:47.734115 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 14:20:47.734115 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 14:20:47.734115 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:20:47.734115 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:47.734115 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 421/832] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jun 7 14:20:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:47.734115 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc158ba-c1dc-4354-958d-9cf4528cd931 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:47.782605 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:47.845984 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8998c96d-a8b5-4c06-8111-97d402c78722 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:47.846290 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc158ba-c1dc-4354-958d-9cf4528cd931 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:47.894051 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba26047-6200-4b30-8f4e-29a1471e01d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:47.894272 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb29956-0d84-4b4a-b110-385a145679a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:47.894983 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb29956-0d84-4b4a-b110-385a145679a1 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:47.895094 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba26047-6200-4b30-8f4e-29a1471e01d7 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:47.895440 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb29956-0d84-4b4a-b110-385a145679a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:47.895548 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba26047-6200-4b30-8f4e-29a1471e01d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:47.895887 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba26047-6200-4b30-8f4e-29a1471e01d7 None admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:47.895991 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb29956-0d84-4b4a-b110-385a145679a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:47.897757 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba26047-6200-4b30-8f4e-29a1471e01d7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:47.897931 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb29956-0d84-4b4a-b110-385a145679a1 None admin] RBAC: Adding query filter params (role.id=9baff2b18f2649f7b757ff2e1f63c853, scope.project.id=724785eb74e44daf8736fd01fc0d1839, user.id=8e857db4902e42989d0d040bbc8517c8) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:47.898262 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba26047-6200-4b30-8f4e-29a1471e01d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:47.898440 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb29956-0d84-4b4a-b110-385a145679a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:47.900593 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba26047-6200-4b30-8f4e-29a1471e01d7 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:47.900853 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb29956-0d84-4b4a-b110-385a145679a1 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:47.920423 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 412/833] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jun 7 14:20:47 2022] GET /identity/v3/role_assignments?user.id=8e857db4902e42989d0d040bbc8517c8&scope.project.id=724785eb74e44daf8736fd01fc0d1839&role.id=9baff2b18f2649f7b757ff2e1f63c853 => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:47.920423 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb29956-0d84-4b4a-b110-385a145679a1 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:47.924441 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 422/834] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jun 7 14:20:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:47.930014 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba26047-6200-4b30-8f4e-29a1471e01d7 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:47.963150 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:48.059548 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb29956-0d84-4b4a-b110-385a145679a1 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:48.102406 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0bb234-8104-4cc0-8fa9-2749cb50a0e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:48.103094 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0bb234-8104-4cc0-8fa9-2749cb50a0e1 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:48.103384 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0bb234-8104-4cc0-8fa9-2749cb50a0e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:48.103674 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0bb234-8104-4cc0-8fa9-2749cb50a0e1 None admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:48.107062 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0bb234-8104-4cc0-8fa9-2749cb50a0e1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:48.107062 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0bb234-8104-4cc0-8fa9-2749cb50a0e1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:48.107987 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0bb234-8104-4cc0-8fa9-2749cb50a0e1 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:48.128414 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 413/835] 10.222.0.85 () {56 vars in 1148 bytes} [Tue Jun 7 14:20:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:48.132950 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0bb234-8104-4cc0-8fa9-2749cb50a0e1 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:48.154555 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba26047-6200-4b30-8f4e-29a1471e01d7 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:48.182514 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:48.218680 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2266d21-520e-4228-a9e5-cdca548c1b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:48.220037 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2266d21-520e-4228-a9e5-cdca548c1b4e None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:48.220545 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2266d21-520e-4228-a9e5-cdca548c1b4e None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:48.222079 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2266d21-520e-4228-a9e5-cdca548c1b4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:48.237366 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2266d21-520e-4228-a9e5-cdca548c1b4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:48.239908 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:48.242326 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2266d21-520e-4228-a9e5-cdca548c1b4e None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:48.255219 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0bb234-8104-4cc0-8fa9-2749cb50a0e1 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:48.264833 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-b2266d21-520e-4228-a9e5-cdca548c1b4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:48.264833 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:48.264833 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:48.264833 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:48.264833 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:48.264833 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:48.264833 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:48.264833 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:48.264833 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:48.264833 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:48.264833 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:48.264833 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:48.264833 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:20:48.264833 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:20:48.264833 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:20:48.270793 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:20:48.270793 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:48.270793 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:48.270793 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:48.270793 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:48.270793 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:48.270793 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:48.270793 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:48.270793 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:48.270793 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:48.270793 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:48.270793 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:48.270793 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:48.270793 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:48.270793 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:48.270793 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:48.270793 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:48.271662 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:20:48.271662 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:20:48.271662 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:48.271662 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:48.272524 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 423/836] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:20:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:48.288532 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2266d21-520e-4228-a9e5-cdca548c1b4e None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:48.324865 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aadb97f-5522-43ff-a175-96238f8daac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:48.326431 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aadb97f-5522-43ff-a175-96238f8daac8 None admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:48.326910 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aadb97f-5522-43ff-a175-96238f8daac8 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:48.327543 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aadb97f-5522-43ff-a175-96238f8daac8 None admin] PATH_INFO: `/v3/projects/724785eb74e44daf8736fd01fc0d1839/users/ad9d021b171244d796c320aa9f8450a9/roles/17516844e8b947b389d62f50718b852d` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:48.355746 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aadb97f-5522-43ff-a175-96238f8daac8 None admin] RBAC: Authorizing `identity:create_grant(project_id=724785eb74e44daf8736fd01fc0d1839, user_id=ad9d021b171244d796c320aa9f8450a9, role_id=17516844e8b947b389d62f50718b852d)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:48.358702 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aadb97f-5522-43ff-a175-96238f8daac8 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:48.407281 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2266d21-520e-4228-a9e5-cdca548c1b4e None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:48.449376 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 414/837] 10.222.0.85 () {58 vars in 1439 bytes} [Tue Jun 7 14:20:48 2022] PUT /identity/v3/projects/724785eb74e44daf8736fd01fc0d1839/users/ad9d021b171244d796c320aa9f8450a9/roles/17516844e8b947b389d62f50718b852d => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:20:48.451133 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad90d034-d584-4094-8bfb-885cfa52f817 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:48.451601 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad90d034-d584-4094-8bfb-885cfa52f817 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:48.452045 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad90d034-d584-4094-8bfb-885cfa52f817 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:48.454036 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 415/838] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:20:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:20:48.454220 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ba29fd-ba5e-41a9-8449-3cffac20841e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:48.455024 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ba29fd-ba5e-41a9-8449-3cffac20841e None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:48.455387 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ba29fd-ba5e-41a9-8449-3cffac20841e None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:48.455742 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ba29fd-ba5e-41a9-8449-3cffac20841e None admin] PATH_INFO: `/v3/domains` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:48.456614 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ba29fd-ba5e-41a9-8449-3cffac20841e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:48.457041 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ba29fd-ba5e-41a9-8449-3cffac20841e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:48.458578 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ba29fd-ba5e-41a9-8449-3cffac20841e None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:48.467411 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aefd39-3387-41fd-8438-0720ee3fe05b None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:48.467708 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aefd39-3387-41fd-8438-0720ee3fe05b None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:48.468161 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aefd39-3387-41fd-8438-0720ee3fe05b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:48.474589 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 424/839] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jun 7 14:20:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:48.480683 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ba29fd-ba5e-41a9-8449-3cffac20841e None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:48.488835 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-c4aefd39-3387-41fd-8438-0720ee3fe05b None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:20:48.490151 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-c4aefd39-3387-41fd-8438-0720ee3fe05b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:20:48.523681 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:48.606014 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 416/840] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:20:48 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 14:20:48.633186 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aefd39-3387-41fd-8438-0720ee3fe05b None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:48.637776 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ba29fd-ba5e-41a9-8449-3cffac20841e None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:48.681527 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba29a5c-1856-4bdb-adf4-150e8bddbee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:48.682054 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba29a5c-1856-4bdb-adf4-150e8bddbee8 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:48.682347 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba29a5c-1856-4bdb-adf4-150e8bddbee8 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:48.682627 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba29a5c-1856-4bdb-adf4-150e8bddbee8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:48.691045 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba29a5c-1856-4bdb-adf4-150e8bddbee8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:48.693306 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba29a5c-1856-4bdb-adf4-150e8bddbee8 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:48.713800 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-aba29a5c-1856-4bdb-adf4-150e8bddbee8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 07 14:20:48.713800 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:48.713800 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:48.713800 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:48.713800 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:48.713800 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:48.713800 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:48.713800 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:48.713800 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:48.713800 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:48.713800 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:48.713800 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:48.713800 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 14:20:48.713800 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 14:20:48.713800 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 14:20:48.713800 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 14:20:48.715233 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:48.715233 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:48.715233 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 14:20:48.715233 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:20:48.715233 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 14:20:48.715233 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:20:48.715233 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:48.715233 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:48.715233 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:48.715233 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:48.715233 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:48.715233 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:48.715233 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:48.715233 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:48.715233 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:48.715233 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:48.715233 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:48.715233 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:48.716339 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:48.716339 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:48.716339 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 14:20:48.716339 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 14:20:48.716339 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 14:20:48.716339 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 14:20:48.716339 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 14:20:48.716339 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 14:20:48.716339 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 07 14:20:48.716339 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:48.718420 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 425/841] 10.222.0.85 () {56 vars in 1096 bytes} [Tue Jun 7 14:20:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:48.726276 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba29a5c-1856-4bdb-adf4-150e8bddbee8 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:48.749926 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:48.796362 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aefd39-3387-41fd-8438-0720ee3fe05b None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:48.847701 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bf7543-909d-4cc8-bf48-bb415a36dac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:48.848967 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bf7543-909d-4cc8-bf48-bb415a36dac2 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:48.849477 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bf7543-909d-4cc8-bf48-bb415a36dac2 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:48.850558 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bf7543-909d-4cc8-bf48-bb415a36dac2 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:48.851646 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bf7543-909d-4cc8-bf48-bb415a36dac2 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:48.854915 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bf7543-909d-4cc8-bf48-bb415a36dac2 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:48.859115 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:48.873139 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-f2bf7543-909d-4cc8-bf48-bb415a36dac2 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 07 14:20:48.873139 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:48.873139 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 14:20:48.873139 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 14:20:48.873139 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 14:20:48.873139 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 14:20:48.873139 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 14:20:48.873139 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:20:48.873139 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 07 14:20:48.873139 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:48.873139 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 14:20:48.873139 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:48.873139 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:48.873139 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:48.873139 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:48.873139 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:48.873139 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:48.874738 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:48.874738 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:48.874738 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:48.874738 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:48.874738 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:48.874738 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:48.874738 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 14:20:48.874738 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 14:20:48.874738 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 14:20:48.874738 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 14:20:48.874738 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:48.874738 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:48.874738 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:48.874738 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:48.874738 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:48.874738 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:48.876265 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:48.876265 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:48.876265 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:48.876265 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:48.876265 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:48.876265 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:48.876265 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:48.876265 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:48.876265 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:48.876265 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:48.876265 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 14:20:48.876265 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:20:48.876265 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 07 14:20:48.876265 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:48.878180 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba29a5c-1856-4bdb-adf4-150e8bddbee8 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:48.880153 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 417/842] 10.222.0.85 () {56 vars in 1114 bytes} [Tue Jun 7 14:20:48 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:48.886417 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bf7543-909d-4cc8-bf48-bb415a36dac2 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:48.935375 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2710a24-3dd9-4ace-9218-54b1cabc6235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:48.936782 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2710a24-3dd9-4ace-9218-54b1cabc6235 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:48.937733 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2710a24-3dd9-4ace-9218-54b1cabc6235 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:48.938211 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2710a24-3dd9-4ace-9218-54b1cabc6235 None admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:48.939449 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2710a24-3dd9-4ace-9218-54b1cabc6235 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:48.940139 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2710a24-3dd9-4ace-9218-54b1cabc6235 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:48.943204 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2710a24-3dd9-4ace-9218-54b1cabc6235 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:48.975467 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 426/843] 10.222.0.85 () {56 vars in 1135 bytes} [Tue Jun 7 14:20:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:20:48.976439 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2710a24-3dd9-4ace-9218-54b1cabc6235 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:49.072691 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bf7543-909d-4cc8-bf48-bb415a36dac2 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:49.118552 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:49.125950 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0b41f5-df8c-41f8-bf92-c5c71b5dd212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:49.126145 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2710a24-3dd9-4ace-9218-54b1cabc6235 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:49.127071 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0b41f5-df8c-41f8-bf92-c5c71b5dd212 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:49.127810 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0b41f5-df8c-41f8-bf92-c5c71b5dd212 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:49.128306 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0b41f5-df8c-41f8-bf92-c5c71b5dd212 None admin] PATH_INFO: `/v3/services` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:49.133642 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0b41f5-df8c-41f8-bf92-c5c71b5dd212 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:49.133642 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0b41f5-df8c-41f8-bf92-c5c71b5dd212 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:49.133642 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0b41f5-df8c-41f8-bf92-c5c71b5dd212 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:49.139032 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 418/844] 10.222.0.85 () {56 vars in 1114 bytes} [Tue Jun 7 14:20:48 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:20:49.146593 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0b41f5-df8c-41f8-bf92-c5c71b5dd212 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:49.188298 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbb7afa-948f-412f-abf7-c28d99e40822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:49.189637 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbb7afa-948f-412f-abf7-c28d99e40822 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:49.190172 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbb7afa-948f-412f-abf7-c28d99e40822 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:49.190664 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbb7afa-948f-412f-abf7-c28d99e40822 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:49.204688 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:49.274970 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbb7afa-948f-412f-abf7-c28d99e40822 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:49.277629 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbb7afa-948f-412f-abf7-c28d99e40822 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:49.279020 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0b41f5-df8c-41f8-bf92-c5c71b5dd212 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:49.333617 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e23ff13-f791-40f7-af52-bb287d566144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:49.335373 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e23ff13-f791-40f7-af52-bb287d566144 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:49.335882 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e23ff13-f791-40f7-af52-bb287d566144 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:49.336397 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e23ff13-f791-40f7-af52-bb287d566144 None admin] PATH_INFO: `/v3/services` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:49.337857 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e23ff13-f791-40f7-af52-bb287d566144 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:49.338454 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e23ff13-f791-40f7-af52-bb287d566144 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:49.341200 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e23ff13-f791-40f7-af52-bb287d566144 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:49.348986 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 419/845] 10.222.0.85 () {56 vars in 1114 bytes} [Tue Jun 7 14:20:49 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:49.415672 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 427/846] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jun 7 14:20:48 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 14:20:49.421652 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e23ff13-f791-40f7-af52-bb287d566144 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:49.496117 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:49.545354 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e23ff13-f791-40f7-af52-bb287d566144 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:49.585560 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45fe2fa-6bdc-4313-bda8-b263014fd29e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:49.586458 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45fe2fa-6bdc-4313-bda8-b263014fd29e None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:49.586758 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45fe2fa-6bdc-4313-bda8-b263014fd29e None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:49.587051 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45fe2fa-6bdc-4313-bda8-b263014fd29e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:49.599189 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45fe2fa-6bdc-4313-bda8-b263014fd29e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:49.600777 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45fe2fa-6bdc-4313-bda8-b263014fd29e None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:49.620906 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-d45fe2fa-6bdc-4313-bda8-b263014fd29e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:49.620906 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:49.620906 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:49.620906 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:49.620906 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:49.620906 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:49.620906 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:49.620906 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:49.620906 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:49.620906 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:49.620906 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:49.620906 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:49.620906 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:20:49.620906 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:20:49.620906 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:20:49.622148 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:20:49.622148 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:49.622148 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:49.622148 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:49.622148 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:49.622148 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:49.622148 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:49.622148 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:49.622148 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:49.622148 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:49.622148 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:49.622148 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:49.622148 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:49.622148 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:49.622148 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:49.622148 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:49.622148 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:49.623297 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:20:49.623297 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:20:49.623297 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:49.623297 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:49.623297 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 420/847] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:20:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:49.629621 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbb7afa-948f-412f-abf7-c28d99e40822 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:49.681891 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:49.747977 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbb7afa-948f-412f-abf7-c28d99e40822 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:49.785911 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b948da5-bd06-4891-810c-375c3638ea46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:49.786599 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b948da5-bd06-4891-810c-375c3638ea46 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:49.786887 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b948da5-bd06-4891-810c-375c3638ea46 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:49.787168 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b948da5-bd06-4891-810c-375c3638ea46 None admin] PATH_INFO: `/v3/domains` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:49.787938 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b948da5-bd06-4891-810c-375c3638ea46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:49.788252 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b948da5-bd06-4891-810c-375c3638ea46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:49.789750 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b948da5-bd06-4891-810c-375c3638ea46 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:49.816731 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 428/848] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jun 7 14:20:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:20:49.828397 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45fe2fa-6bdc-4313-bda8-b263014fd29e None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:49.934061 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45fe2fa-6bdc-4313-bda8-b263014fd29e None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:49.948705 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:49.971863 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9004fd-9c61-4320-ad2e-9e2ea2a117a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:49.972813 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9004fd-9c61-4320-ad2e-9e2ea2a117a6 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:49.988728 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9004fd-9c61-4320-ad2e-9e2ea2a117a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:49.988728 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9004fd-9c61-4320-ad2e-9e2ea2a117a6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:49.988728 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9004fd-9c61-4320-ad2e-9e2ea2a117a6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:49.988728 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9004fd-9c61-4320-ad2e-9e2ea2a117a6 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:49.999009 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-7c9004fd-9c61-4320-ad2e-9e2ea2a117a6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:20:49.999009 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:49.999009 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:49.999009 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:49.999009 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:49.999009 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:49.999009 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:49.999009 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:49.999009 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:49.999009 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:49.999009 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:49.999009 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:49.999009 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 14:20:49.999009 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 14:20:49.999009 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 14:20:49.999929 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 14:20:49.999929 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:49.999929 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:49.999929 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:49.999929 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:49.999929 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:49.999929 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:49.999929 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:49.999929 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:49.999929 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:49.999929 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:49.999929 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:49.999929 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:49.999929 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:49.999929 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:49.999929 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:49.999929 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:50.000829 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 14:20:50.000829 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 14:20:50.000829 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 14:20:50.000829 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 14:20:50.000829 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 14:20:50.000829 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 14:20:50.000829 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:20:50.000829 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:50.000829 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 421/849] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jun 7 14:20:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:50.007059 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b948da5-bd06-4891-810c-375c3638ea46 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:50.127654 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b948da5-bd06-4891-810c-375c3638ea46 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:50.129287 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:50.164476 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6df950-c65e-49f1-978a-c0167490d229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:50.165152 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6df950-c65e-49f1-978a-c0167490d229 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:50.165441 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6df950-c65e-49f1-978a-c0167490d229 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:50.165724 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6df950-c65e-49f1-978a-c0167490d229 None admin] PATH_INFO: `/v3/projects` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:50.166514 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6df950-c65e-49f1-978a-c0167490d229 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:50.166831 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6df950-c65e-49f1-978a-c0167490d229 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:50.168261 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6df950-c65e-49f1-978a-c0167490d229 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:50.188083 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9004fd-9c61-4320-ad2e-9e2ea2a117a6 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:50.198591 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 429/850] 10.222.0.85 () {56 vars in 1148 bytes} [Tue Jun 7 14:20:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:50.306481 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9004fd-9c61-4320-ad2e-9e2ea2a117a6 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:50.344110 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837fa6c1-1e9b-485d-ad02-f8181cc7b230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:50.344803 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837fa6c1-1e9b-485d-ad02-f8181cc7b230 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:50.345135 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837fa6c1-1e9b-485d-ad02-f8181cc7b230 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:50.345442 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837fa6c1-1e9b-485d-ad02-f8181cc7b230 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:50.346324 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837fa6c1-1e9b-485d-ad02-f8181cc7b230 None admin] RBAC: Adding query filter params (role.id=9baff2b18f2649f7b757ff2e1f63c853, scope.project.id=724785eb74e44daf8736fd01fc0d1839, user.id=35aa67025f66459da0c0fc3cbfd45430) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:50.346644 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837fa6c1-1e9b-485d-ad02-f8181cc7b230 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:50.348091 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837fa6c1-1e9b-485d-ad02-f8181cc7b230 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:50.354733 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 422/851] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jun 7 14:20:50 2022] GET /identity/v3/role_assignments?user.id=35aa67025f66459da0c0fc3cbfd45430&scope.project.id=724785eb74e44daf8736fd01fc0d1839&role.id=9baff2b18f2649f7b757ff2e1f63c853 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:50.614605 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fa82b1-e44e-4965-bb57-4f2ea469176a None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:50.614823 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fa82b1-e44e-4965-bb57-4f2ea469176a None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:50.615302 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fa82b1-e44e-4965-bb57-4f2ea469176a None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:50.618115 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 430/852] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:20:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:20:50.624315 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceb8446-af85-4519-aaa4-d767131e9c5c None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:50.624629 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceb8446-af85-4519-aaa4-d767131e9c5c None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:50.624924 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceb8446-af85-4519-aaa4-d767131e9c5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:50.647521 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-aceb8446-af85-4519-aaa4-d767131e9c5c None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:20:50.648796 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-aceb8446-af85-4519-aaa4-d767131e9c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:20:50.687931 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:50.733480 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 423/853] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:20:50 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:20:50.756759 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fa82b1-e44e-4965-bb57-4f2ea469176a None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:50.880946 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fa82b1-e44e-4965-bb57-4f2ea469176a None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:50.920778 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491b8fc5-936a-4b7f-a778-6f4e5875b376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:50.921937 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491b8fc5-936a-4b7f-a778-6f4e5875b376 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:50.922417 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491b8fc5-936a-4b7f-a778-6f4e5875b376 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:50.922916 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491b8fc5-936a-4b7f-a778-6f4e5875b376 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:50.923762 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491b8fc5-936a-4b7f-a778-6f4e5875b376 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:50.925380 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491b8fc5-936a-4b7f-a778-6f4e5875b376 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:50.938353 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-491b8fc5-936a-4b7f-a778-6f4e5875b376 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 07 14:20:50.938353 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:50.938353 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 14:20:50.938353 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 14:20:50.938353 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 14:20:50.938353 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 14:20:50.938353 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 14:20:50.938353 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:20:50.938353 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 07 14:20:50.938353 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:50.938353 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 14:20:50.938353 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:50.938353 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:50.938353 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:50.938353 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:50.938353 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:50.939556 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:50.939556 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:50.939556 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:50.939556 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:50.939556 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:50.939556 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:50.939556 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:50.939556 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 14:20:50.939556 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 14:20:50.939556 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 14:20:50.939556 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 14:20:50.939556 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:50.939556 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:50.939556 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:50.939556 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:50.939556 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:50.940408 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:50.940408 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:50.940408 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:50.940408 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:50.940408 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:50.940408 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:50.940408 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:50.940408 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:50.940408 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:50.940408 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:50.940408 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:50.940408 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 14:20:50.940408 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:20:50.940408 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 07 14:20:50.940408 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:50.948737 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 431/854] 10.222.0.85 () {56 vars in 1132 bytes} [Tue Jun 7 14:20:50 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:50.951867 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceb8446-af85-4519-aaa4-d767131e9c5c None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:51.057653 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5389788e-c9a1-46d8-ad1b-f4959e492194 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:51.057653 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5389788e-c9a1-46d8-ad1b-f4959e492194 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:51.057653 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5389788e-c9a1-46d8-ad1b-f4959e492194 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:51.057653 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 432/855] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:20:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 14:20:51.063068 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7e0a97-57eb-4896-b351-318696937248 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:51.063361 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7e0a97-57eb-4896-b351-318696937248 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:51.063638 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7e0a97-57eb-4896-b351-318696937248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:51.066722 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceb8446-af85-4519-aaa4-d767131e9c5c None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:51.082258 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-8a7e0a97-57eb-4896-b351-318696937248 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:20:51.083453 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-8a7e0a97-57eb-4896-b351-318696937248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:20:51.107129 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:51.108144 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f489390b-f9a1-4ea4-a084-2e6341645a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:51.108920 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f489390b-f9a1-4ea4-a084-2e6341645a2a None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:51.109206 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f489390b-f9a1-4ea4-a084-2e6341645a2a None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:51.109484 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f489390b-f9a1-4ea4-a084-2e6341645a2a None admin] PATH_INFO: `/v3/services` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:51.110210 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f489390b-f9a1-4ea4-a084-2e6341645a2a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:51.110523 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f489390b-f9a1-4ea4-a084-2e6341645a2a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:51.111896 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f489390b-f9a1-4ea4-a084-2e6341645a2a None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:51.117516 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 424/856] 10.222.0.85 () {56 vars in 1126 bytes} [Tue Jun 7 14:20:50 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:51.123713 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f489390b-f9a1-4ea4-a084-2e6341645a2a None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:51.157080 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:51.176397 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 433/857] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:20:51 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:20:51.196287 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7e0a97-57eb-4896-b351-318696937248 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:51.243824 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f489390b-f9a1-4ea4-a084-2e6341645a2a None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:51.284216 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16199eab-df91-4f96-8135-6566afeecbf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:51.285307 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16199eab-df91-4f96-8135-6566afeecbf1 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:51.285803 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16199eab-df91-4f96-8135-6566afeecbf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:51.286280 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16199eab-df91-4f96-8135-6566afeecbf1 None admin] PATH_INFO: `/v3/services` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:51.287509 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16199eab-df91-4f96-8135-6566afeecbf1 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:51.288044 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16199eab-df91-4f96-8135-6566afeecbf1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:51.289691 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16199eab-df91-4f96-8135-6566afeecbf1 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:51.300891 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 425/858] 10.222.0.85 () {56 vars in 1126 bytes} [Tue Jun 7 14:20:51 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:51.322619 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7e0a97-57eb-4896-b351-318696937248 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:51.364938 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc718bf-a2be-436e-8b9e-df7d38545f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:51.366338 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc718bf-a2be-436e-8b9e-df7d38545f76 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:51.366841 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc718bf-a2be-436e-8b9e-df7d38545f76 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:51.367332 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc718bf-a2be-436e-8b9e-df7d38545f76 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:51.383643 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc718bf-a2be-436e-8b9e-df7d38545f76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:20:51.391488 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-9bc718bf-a2be-436e-8b9e-df7d38545f76 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:20:51.391488 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:51.391488 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:51.391488 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:51.391488 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:51.391488 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:51.391488 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:51.391488 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:51.391488 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:51.391488 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:51.391488 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:51.391488 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:51.391488 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:20:51.391488 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:20:51.391488 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:20:51.391488 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:20:51.392505 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:20:51.392505 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:20:51.392505 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:20:51.392505 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:20:51.392505 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:51.392505 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:51.392505 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:51.392505 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:51.392505 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:51.392505 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:51.392505 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:51.392505 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:51.392505 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:51.392505 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:51.392505 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:51.392505 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:51.393572 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:51.393572 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:51.393572 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:51.393572 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:51.393572 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:20:51.393572 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:20:51.393572 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:20:51.393572 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:20:51.393572 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:20:51.393572 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:20:51.393572 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:20:51.393572 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:51.393572 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 434/859] 10.222.0.85 () {56 vars in 1102 bytes} [Tue Jun 7 14:20:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:20:51.401841 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16199eab-df91-4f96-8135-6566afeecbf1 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:51.519759 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:51.527268 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16199eab-df91-4f96-8135-6566afeecbf1 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:51.569920 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781a2bba-e10a-4a6e-8b48-7e09a6256da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:51.570600 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781a2bba-e10a-4a6e-8b48-7e09a6256da5 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:51.570888 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781a2bba-e10a-4a6e-8b48-7e09a6256da5 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:51.571171 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781a2bba-e10a-4a6e-8b48-7e09a6256da5 None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:51.571928 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781a2bba-e10a-4a6e-8b48-7e09a6256da5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:51.572243 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781a2bba-e10a-4a6e-8b48-7e09a6256da5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:51.573651 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781a2bba-e10a-4a6e-8b48-7e09a6256da5 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:51.582829 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 426/860] 10.222.0.85 () {56 vars in 1103 bytes} [Tue Jun 7 14:20:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:51.588568 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc718bf-a2be-436e-8b9e-df7d38545f76 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:51.610039 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:51.713314 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc718bf-a2be-436e-8b9e-df7d38545f76 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:51.752531 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1842c258-e811-495f-a6fb-6380f7bd2371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:51.753336 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1842c258-e811-495f-a6fb-6380f7bd2371 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:51.753724 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1842c258-e811-495f-a6fb-6380f7bd2371 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:51.754014 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1842c258-e811-495f-a6fb-6380f7bd2371 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:51.765642 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1842c258-e811-495f-a6fb-6380f7bd2371 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:51.768993 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1842c258-e811-495f-a6fb-6380f7bd2371 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:51.784804 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-1842c258-e811-495f-a6fb-6380f7bd2371 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:51.784804 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:51.784804 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:51.784804 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:51.784804 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:51.784804 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:51.784804 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:51.784804 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:51.784804 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:51.784804 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:51.784804 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:51.784804 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:51.784804 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:20:51.784804 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:20:51.784804 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:20:51.786945 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:20:51.786945 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:51.786945 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:51.786945 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:51.786945 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:51.786945 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:51.786945 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:51.786945 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:51.786945 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:51.786945 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:51.786945 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:51.786945 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:51.786945 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:51.786945 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:51.786945 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:51.786945 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:51.786945 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:51.789691 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:20:51.789691 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:20:51.789691 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:51.789691 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:51.789691 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 435/861] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:20:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:51.796127 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781a2bba-e10a-4a6e-8b48-7e09a6256da5 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:51.823888 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fc34cc-6547-47ba-8488-87f85fd57a47 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:51.824232 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fc34cc-6547-47ba-8488-87f85fd57a47 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:51.824483 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fc34cc-6547-47ba-8488-87f85fd57a47 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:51.826168 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 436/862] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:20:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:20:51.833927 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6790db49-645b-4acb-af90-8bc7344ac0f1 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:51.834377 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6790db49-645b-4acb-af90-8bc7344ac0f1 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:51.834729 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6790db49-645b-4acb-af90-8bc7344ac0f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:51.857910 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-6790db49-645b-4acb-af90-8bc7344ac0f1 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:20:51.859490 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-6790db49-645b-4acb-af90-8bc7344ac0f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:20:51.915925 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781a2bba-e10a-4a6e-8b48-7e09a6256da5 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:51.950039 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:51.953366 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381c6b3c-69c0-4bb6-a548-30a0964eb275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:51.954636 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c6b3c-69c0-4bb6-a548-30a0964eb275 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:51.955133 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c6b3c-69c0-4bb6-a548-30a0964eb275 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:51.955612 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c6b3c-69c0-4bb6-a548-30a0964eb275 None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:51.956589 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381c6b3c-69c0-4bb6-a548-30a0964eb275 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:51.956826 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 437/863] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:20:51 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:20:51.957354 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381c6b3c-69c0-4bb6-a548-30a0964eb275 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:51.960865 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381c6b3c-69c0-4bb6-a548-30a0964eb275 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:51.982172 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 427/864] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jun 7 14:20:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:51.982968 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6790db49-645b-4acb-af90-8bc7344ac0f1 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:51.984306 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381c6b3c-69c0-4bb6-a548-30a0964eb275 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:52.003999 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:52.092995 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6790db49-645b-4acb-af90-8bc7344ac0f1 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:52.095648 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381c6b3c-69c0-4bb6-a548-30a0964eb275 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:52.131113 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2bfbdd-1b6b-4dd3-96bb-0873fb4c68f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:52.131794 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2bfbdd-1b6b-4dd3-96bb-0873fb4c68f8 None admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:52.132082 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2bfbdd-1b6b-4dd3-96bb-0873fb4c68f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:52.132363 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2bfbdd-1b6b-4dd3-96bb-0873fb4c68f8 None admin] PATH_INFO: `/v3/services` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:52.133373 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2bfbdd-1b6b-4dd3-96bb-0873fb4c68f8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:52.134288 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58941ff4-5975-4e81-9482-f7320252a0ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:52.134745 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2bfbdd-1b6b-4dd3-96bb-0873fb4c68f8 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:52.135525 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58941ff4-5975-4e81-9482-f7320252a0ca None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:52.135949 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58941ff4-5975-4e81-9482-f7320252a0ca None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:52.136365 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58941ff4-5975-4e81-9482-f7320252a0ca None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:52.145777 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58941ff4-5975-4e81-9482-f7320252a0ca None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:52.147818 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58941ff4-5975-4e81-9482-f7320252a0ca None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:52.165075 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-58941ff4-5975-4e81-9482-f7320252a0ca None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 14:20:52.165075 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:52.165075 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:52.165075 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:52.165075 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:52.165075 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:52.165075 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:52.165075 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:52.165075 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:52.165075 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:52.165075 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:52.165075 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:52.165075 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 14:20:52.165075 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 14:20:52.165075 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 14:20:52.165075 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 14:20:52.166212 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:52.166212 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:52.166212 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 14:20:52.166212 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:20:52.166212 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 14:20:52.166212 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:20:52.166212 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:52.166212 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:52.166212 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:52.166212 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:52.166212 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:52.166212 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:52.166212 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:52.166212 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:52.166212 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:52.166212 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:52.166212 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:52.166212 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:52.167173 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:52.167173 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:52.167173 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 14:20:52.167173 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 14:20:52.167173 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 14:20:52.167173 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 14:20:52.167173 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 14:20:52.167173 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 14:20:52.167173 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 14:20:52.167173 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:52.170973 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 428/865] 10.222.0.85 () {56 vars in 1117 bytes} [Tue Jun 7 14:20:51 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:52.172255 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58941ff4-5975-4e81-9482-f7320252a0ca None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:52.215847 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 438/866] 10.222.0.85 () {60 vars in 1141 bytes} [Tue Jun 7 14:20:51 2022] POST /identity/v3/services => generated 251 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:20:52.279015 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:52.308122 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58941ff4-5975-4e81-9482-f7320252a0ca None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:52.363862 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a657db7a-ec88-4bc1-a514-c9c7b89708f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:52.364518 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a657db7a-ec88-4bc1-a514-c9c7b89708f3 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:52.364950 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a657db7a-ec88-4bc1-a514-c9c7b89708f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:52.365234 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a657db7a-ec88-4bc1-a514-c9c7b89708f3 None admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:52.366152 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a657db7a-ec88-4bc1-a514-c9c7b89708f3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:52.366472 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a657db7a-ec88-4bc1-a514-c9c7b89708f3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:52.367982 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a657db7a-ec88-4bc1-a514-c9c7b89708f3 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:52.391015 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 429/867] 10.222.0.85 () {56 vars in 1149 bytes} [Tue Jun 7 14:20:52 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:52.397548 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2bfbdd-1b6b-4dd3-96bb-0873fb4c68f8 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:52.468156 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:52.517945 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2bfbdd-1b6b-4dd3-96bb-0873fb4c68f8 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:52.555364 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca7dc2b-f13b-4d46-8d46-890fc25f87b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:52.556451 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca7dc2b-f13b-4d46-8d46-890fc25f87b5 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:52.556960 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca7dc2b-f13b-4d46-8d46-890fc25f87b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:52.557456 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca7dc2b-f13b-4d46-8d46-890fc25f87b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:52.632260 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca7dc2b-f13b-4d46-8d46-890fc25f87b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:52.634072 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca7dc2b-f13b-4d46-8d46-890fc25f87b5 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:52.696064 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:52.746134 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 439/868] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jun 7 14:20:52 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 14:20:52.747574 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a657db7a-ec88-4bc1-a514-c9c7b89708f3 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:52.820031 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ae7526-347e-48fa-8d33-ce36e1180f7e None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:52.820697 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ae7526-347e-48fa-8d33-ce36e1180f7e None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:52.820697 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ae7526-347e-48fa-8d33-ce36e1180f7e None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:52.822706 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 440/869] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:20:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 14:20:52.830535 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda878a5-0a4a-4df8-9faf-4e2e984a0a16 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:52.830967 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda878a5-0a4a-4df8-9faf-4e2e984a0a16 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:52.831496 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda878a5-0a4a-4df8-9faf-4e2e984a0a16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:52.849374 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-eda878a5-0a4a-4df8-9faf-4e2e984a0a16 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:20:52.850731 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-eda878a5-0a4a-4df8-9faf-4e2e984a0a16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:20:52.876490 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a657db7a-ec88-4bc1-a514-c9c7b89708f3 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:52.909916 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:52.916180 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec94e55-e9c5-4f64-8cd6-4ee3f862869c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:52.923767 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec94e55-e9c5-4f64-8cd6-4ee3f862869c None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:52.923767 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec94e55-e9c5-4f64-8cd6-4ee3f862869c None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:52.923767 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec94e55-e9c5-4f64-8cd6-4ee3f862869c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:52.930104 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec94e55-e9c5-4f64-8cd6-4ee3f862869c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:52.931907 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec94e55-e9c5-4f64-8cd6-4ee3f862869c None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:52.932486 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 441/870] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:20:52 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 14:20:52.950519 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda878a5-0a4a-4df8-9faf-4e2e984a0a16 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:52.955687 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-1ec94e55-e9c5-4f64-8cd6-4ee3f862869c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:52.955687 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:52.955687 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:52.955687 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:52.955687 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:52.955687 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:52.955687 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:52.955687 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:52.955687 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:52.955687 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:52.955687 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:52.955687 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:52.955687 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:20:52.955687 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:20:52.955687 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:20:52.956624 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:20:52.956624 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:52.956624 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:52.956624 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:52.956624 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:52.956624 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:52.956624 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:52.956624 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:52.956624 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:52.956624 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:52.956624 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:52.956624 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:52.956624 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:52.956624 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:52.956624 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:52.956624 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:52.956624 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:52.957531 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:20:52.957531 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:20:52.957531 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:52.957531 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:52.959881 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 430/871] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:20:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:52.965492 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec94e55-e9c5-4f64-8cd6-4ee3f862869c None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:53.054454 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:53.067137 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda878a5-0a4a-4df8-9faf-4e2e984a0a16 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:53.092556 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec94e55-e9c5-4f64-8cd6-4ee3f862869c None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:53.103633 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d401e1-92fa-484d-bd76-abec731b3da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:53.104495 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d401e1-92fa-484d-bd76-abec731b3da3 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:53.105053 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d401e1-92fa-484d-bd76-abec731b3da3 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:53.105053 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d401e1-92fa-484d-bd76-abec731b3da3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:53.105709 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d401e1-92fa-484d-bd76-abec731b3da3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:53.107119 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d401e1-92fa-484d-bd76-abec731b3da3 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:53.131864 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-97d401e1-92fa-484d-bd76-abec731b3da3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 14:20:53.131864 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:53.131864 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 14:20:53.131864 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 14:20:53.131864 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 14:20:53.131864 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 14:20:53.131864 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 14:20:53.131864 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:20:53.131864 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 14:20:53.131864 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:53.131864 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 14:20:53.131864 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:53.131864 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:53.131864 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:53.131864 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:53.131864 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:53.131864 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:53.132958 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:53.132958 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:53.132958 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:53.132958 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:53.132958 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:53.132958 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:53.132958 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 14:20:53.132958 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 14:20:53.132958 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 14:20:53.132958 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 14:20:53.132958 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:53.132958 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:53.132958 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:53.132958 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:53.132958 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:53.132958 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:53.133842 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:53.133842 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:53.133842 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:53.133842 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:53.133842 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:53.133842 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:53.133842 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:53.133842 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:53.133842 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:53.133842 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:53.133842 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 14:20:53.133842 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:20:53.133842 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 14:20:53.133842 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:53.141279 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 442/872] 10.222.0.85 () {56 vars in 1126 bytes} [Tue Jun 7 14:20:52 2022] GET /identity/v3/services/object-store => generated 93 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:53.142261 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d401e1-92fa-484d-bd76-abec731b3da3 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:53.158730 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468038d5-ed8a-458d-b2dd-26c2777dde54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:53.160820 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468038d5-ed8a-458d-b2dd-26c2777dde54 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:53.161813 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468038d5-ed8a-458d-b2dd-26c2777dde54 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:53.163088 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468038d5-ed8a-458d-b2dd-26c2777dde54 None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:53.164111 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468038d5-ed8a-458d-b2dd-26c2777dde54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:53.164709 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468038d5-ed8a-458d-b2dd-26c2777dde54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:53.166742 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468038d5-ed8a-458d-b2dd-26c2777dde54 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:53.186594 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 431/873] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jun 7 14:20:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:20:53.193873 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468038d5-ed8a-458d-b2dd-26c2777dde54 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:53.251923 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:53.258964 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d401e1-92fa-484d-bd76-abec731b3da3 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:53.295729 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdbdbb8-660d-4432-871f-2440f6237f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:53.296398 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdbdbb8-660d-4432-871f-2440f6237f35 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:53.296715 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdbdbb8-660d-4432-871f-2440f6237f35 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:53.297261 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdbdbb8-660d-4432-871f-2440f6237f35 None admin] PATH_INFO: `/v3/services` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:53.297777 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdbdbb8-660d-4432-871f-2440f6237f35 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:53.298095 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdbdbb8-660d-4432-871f-2440f6237f35 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:53.299542 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdbdbb8-660d-4432-871f-2440f6237f35 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:53.309982 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 443/874] 10.222.0.85 () {56 vars in 1122 bytes} [Tue Jun 7 14:20:53 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:53.311412 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdbdbb8-660d-4432-871f-2440f6237f35 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:53.324906 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468038d5-ed8a-458d-b2dd-26c2777dde54 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:53.331040 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:53.364516 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f087ed2c-26dc-4b5c-a79e-ecf0f985737b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:53.366556 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f087ed2c-26dc-4b5c-a79e-ecf0f985737b None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:53.367066 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f087ed2c-26dc-4b5c-a79e-ecf0f985737b None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:53.367572 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f087ed2c-26dc-4b5c-a79e-ecf0f985737b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:53.377100 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f087ed2c-26dc-4b5c-a79e-ecf0f985737b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:53.379015 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f087ed2c-26dc-4b5c-a79e-ecf0f985737b None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:53.398005 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-f087ed2c-26dc-4b5c-a79e-ecf0f985737b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:20:53.398005 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:53.398005 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:53.398005 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:53.398005 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:53.398005 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:53.398005 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:53.398005 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:53.398005 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:53.398005 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:53.398005 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:53.398005 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:53.398005 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 14:20:53.398005 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 14:20:53.398005 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 14:20:53.398996 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 14:20:53.398996 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:53.398996 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:53.398996 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:53.398996 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:53.398996 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:53.398996 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:53.398996 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:53.398996 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:53.398996 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:53.398996 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:53.398996 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:53.398996 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:53.398996 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:53.398996 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:53.398996 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:53.398996 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:53.399881 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 14:20:53.399881 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 14:20:53.399881 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 14:20:53.399881 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 14:20:53.399881 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 14:20:53.399881 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 14:20:53.399881 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:20:53.399881 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:53.408524 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 432/875] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jun 7 14:20:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:20:53.409831 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f087ed2c-26dc-4b5c-a79e-ecf0f985737b None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:53.437012 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdbdbb8-660d-4432-871f-2440f6237f35 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:53.477658 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4beb4a-bd65-43d3-a47e-5d9fd3026c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:53.478337 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4beb4a-bd65-43d3-a47e-5d9fd3026c45 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:53.478622 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4beb4a-bd65-43d3-a47e-5d9fd3026c45 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:53.478900 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4beb4a-bd65-43d3-a47e-5d9fd3026c45 None admin] PATH_INFO: `/v3/services` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:53.479647 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4beb4a-bd65-43d3-a47e-5d9fd3026c45 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:53.479916 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4beb4a-bd65-43d3-a47e-5d9fd3026c45 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:53.481327 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4beb4a-bd65-43d3-a47e-5d9fd3026c45 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:53.492636 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 444/876] 10.222.0.85 () {56 vars in 1122 bytes} [Tue Jun 7 14:20:53 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:53.528500 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f087ed2c-26dc-4b5c-a79e-ecf0f985737b None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:53.530693 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:53.569617 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72066fb-3cc2-46a9-8e2b-61c3d708e738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:53.570718 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72066fb-3cc2-46a9-8e2b-61c3d708e738 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:53.571208 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72066fb-3cc2-46a9-8e2b-61c3d708e738 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:53.571804 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72066fb-3cc2-46a9-8e2b-61c3d708e738 None admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:53.574114 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72066fb-3cc2-46a9-8e2b-61c3d708e738 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:53.574697 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72066fb-3cc2-46a9-8e2b-61c3d708e738 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:53.576410 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72066fb-3cc2-46a9-8e2b-61c3d708e738 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:53.591074 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 433/877] 10.222.0.85 () {56 vars in 1148 bytes} [Tue Jun 7 14:20:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:53.597608 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4beb4a-bd65-43d3-a47e-5d9fd3026c45 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:53.697605 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:53.719542 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4beb4a-bd65-43d3-a47e-5d9fd3026c45 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:53.723623 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b2102-7a1d-4bda-b73d-4ae8fbb85c1d None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:53.723914 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b2102-7a1d-4bda-b73d-4ae8fbb85c1d None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:53.724196 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b2102-7a1d-4bda-b73d-4ae8fbb85c1d None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:53.726018 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 434/878] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:20:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 14:20:53.732921 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1009bc9-efd5-4784-bf1e-34c696a39db8 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:53.733210 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1009bc9-efd5-4784-bf1e-34c696a39db8 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:53.733495 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1009bc9-efd5-4784-bf1e-34c696a39db8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:53.759509 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-f1009bc9-efd5-4784-bf1e-34c696a39db8 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:20:53.761090 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-f1009bc9-efd5-4784-bf1e-34c696a39db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:20:53.775448 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f3d12f-4f13-43d5-b179-f6cfefbdda18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:53.776122 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f3d12f-4f13-43d5-b179-f6cfefbdda18 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:53.776681 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f3d12f-4f13-43d5-b179-f6cfefbdda18 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:53.776681 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f3d12f-4f13-43d5-b179-f6cfefbdda18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:53.777531 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f3d12f-4f13-43d5-b179-f6cfefbdda18 None admin] RBAC: Adding query filter params (role.id=17516844e8b947b389d62f50718b852d, scope.project.id=724785eb74e44daf8736fd01fc0d1839, user.id=ad9d021b171244d796c320aa9f8450a9) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:53.777840 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f3d12f-4f13-43d5-b179-f6cfefbdda18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:53.779481 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f3d12f-4f13-43d5-b179-f6cfefbdda18 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:53.789074 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 445/879] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jun 7 14:20:53 2022] GET /identity/v3/role_assignments?user.id=ad9d021b171244d796c320aa9f8450a9&scope.project.id=724785eb74e44daf8736fd01fc0d1839&role.id=17516844e8b947b389d62f50718b852d => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:53.867349 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 435/880] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:20:53 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 14:20:53.891381 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f3d12f-4f13-43d5-b179-f6cfefbdda18 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:54.007963 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f3d12f-4f13-43d5-b179-f6cfefbdda18 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:54.050951 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad00ef2-fb39-45ce-800b-2439d873e199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:54.051753 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad00ef2-fb39-45ce-800b-2439d873e199 None admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:54.052043 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad00ef2-fb39-45ce-800b-2439d873e199 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:54.052398 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad00ef2-fb39-45ce-800b-2439d873e199 None admin] PATH_INFO: `/v3/services` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:54.053703 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad00ef2-fb39-45ce-800b-2439d873e199 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:54.055084 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad00ef2-fb39-45ce-800b-2439d873e199 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:54.189708 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 446/881] 10.222.0.85 () {60 vars in 1141 bytes} [Tue Jun 7 14:20:53 2022] POST /identity/v3/services => generated 269 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:20:54.625153 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3c7d06-9655-428a-8895-fe9b95d0030d None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:54.625711 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3c7d06-9655-428a-8895-fe9b95d0030d None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:54.625711 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3c7d06-9655-428a-8895-fe9b95d0030d None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:54.627365 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 436/882] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:20:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:20:54.633728 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988f06bd-9a00-4ad4-bcf0-469afc89fe35 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:54.634015 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988f06bd-9a00-4ad4-bcf0-469afc89fe35 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:54.634301 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988f06bd-9a00-4ad4-bcf0-469afc89fe35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:54.642519 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:54.654345 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-988f06bd-9a00-4ad4-bcf0-469afc89fe35 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:20:54.656062 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-988f06bd-9a00-4ad4-bcf0-469afc89fe35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:20:54.740181 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 447/883] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:20:54 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:20:54.764837 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3c7d06-9655-428a-8895-fe9b95d0030d None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:54.870118 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:54.880131 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3c7d06-9655-428a-8895-fe9b95d0030d None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:54.917944 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5ebd77-e208-472b-a9cd-9464ba3fb503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:54.918758 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5ebd77-e208-472b-a9cd-9464ba3fb503 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:54.919044 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5ebd77-e208-472b-a9cd-9464ba3fb503 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:54.919318 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5ebd77-e208-472b-a9cd-9464ba3fb503 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:54.920059 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5ebd77-e208-472b-a9cd-9464ba3fb503 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:54.921590 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5ebd77-e208-472b-a9cd-9464ba3fb503 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:54.934986 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-3d5ebd77-e208-472b-a9cd-9464ba3fb503 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 07 14:20:54.934986 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:54.934986 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 14:20:54.934986 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 14:20:54.934986 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 14:20:54.934986 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 14:20:54.934986 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 14:20:54.934986 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:20:54.934986 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 07 14:20:54.934986 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:54.934986 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 14:20:54.934986 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:54.934986 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:54.934986 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:54.934986 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:54.934986 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:54.934986 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:54.936050 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:54.936050 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:54.936050 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:54.936050 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:54.936050 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:54.936050 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:54.936050 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 14:20:54.936050 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 14:20:54.936050 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 14:20:54.936050 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 14:20:54.936050 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:54.936050 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:54.936050 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:54.936050 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:54.936050 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:54.936050 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:54.936961 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:54.936961 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:54.936961 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:54.936961 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:54.936961 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:54.936961 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:54.936961 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:54.936961 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:54.936961 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:54.936961 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:54.936961 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 14:20:54.936961 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:20:54.936961 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 07 14:20:54.936961 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:54.936961 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 437/884] 10.222.0.85 () {56 vars in 1114 bytes} [Tue Jun 7 14:20:54 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:54.942935 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988f06bd-9a00-4ad4-bcf0-469afc89fe35 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:55.064762 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988f06bd-9a00-4ad4-bcf0-469afc89fe35 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:55.098212 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:55.105856 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843c813b-96c6-4ff5-a5ea-c5a0126f8fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:55.111065 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843c813b-96c6-4ff5-a5ea-c5a0126f8fab None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:55.111065 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843c813b-96c6-4ff5-a5ea-c5a0126f8fab None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:55.111065 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843c813b-96c6-4ff5-a5ea-c5a0126f8fab None admin] PATH_INFO: `/v3/services` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:55.111065 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843c813b-96c6-4ff5-a5ea-c5a0126f8fab None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:55.111065 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843c813b-96c6-4ff5-a5ea-c5a0126f8fab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:55.111065 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843c813b-96c6-4ff5-a5ea-c5a0126f8fab None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:55.115418 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 448/885] 10.222.0.85 () {56 vars in 1114 bytes} [Tue Jun 7 14:20:54 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:55.121416 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5ebd77-e208-472b-a9cd-9464ba3fb503 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:55.241020 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5ebd77-e208-472b-a9cd-9464ba3fb503 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:55.278173 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b548a9-9b2d-410f-8df2-e9d859d93fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:55.278849 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b548a9-9b2d-410f-8df2-e9d859d93fff None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:55.279137 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b548a9-9b2d-410f-8df2-e9d859d93fff None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:55.279416 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b548a9-9b2d-410f-8df2-e9d859d93fff None admin] PATH_INFO: `/v3/services` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:55.280134 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b548a9-9b2d-410f-8df2-e9d859d93fff None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:55.280472 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b548a9-9b2d-410f-8df2-e9d859d93fff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:55.281846 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b548a9-9b2d-410f-8df2-e9d859d93fff None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:55.287233 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 438/886] 10.222.0.85 () {56 vars in 1114 bytes} [Tue Jun 7 14:20:55 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:55.292957 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843c813b-96c6-4ff5-a5ea-c5a0126f8fab None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:55.409080 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843c813b-96c6-4ff5-a5ea-c5a0126f8fab None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:55.450858 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1aee3f-3ed9-467a-b553-7bd4dbb7eb22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:55.452007 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1aee3f-3ed9-467a-b553-7bd4dbb7eb22 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:55.452524 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1aee3f-3ed9-467a-b553-7bd4dbb7eb22 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:55.452980 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1aee3f-3ed9-467a-b553-7bd4dbb7eb22 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:55.454404 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1aee3f-3ed9-467a-b553-7bd4dbb7eb22 None admin] RBAC: Adding query filter params (interface=public, service_id=912892b6673b47269458649c1f589430, region_id=RegionOne) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:55.455014 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1aee3f-3ed9-467a-b553-7bd4dbb7eb22 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:55.457568 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1aee3f-3ed9-467a-b553-7bd4dbb7eb22 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:55.468067 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 449/887] 10.222.0.85 () {56 vars in 1251 bytes} [Tue Jun 7 14:20:55 2022] GET /identity/v3/endpoints?service_id=912892b6673b47269458649c1f589430&interface=public®ion_id=RegionOne => generated 194 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:55.473708 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b548a9-9b2d-410f-8df2-e9d859d93fff None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:55.498159 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:55.589601 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b548a9-9b2d-410f-8df2-e9d859d93fff None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:55.627796 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3504022-cc27-4fbe-b87c-eec5dbc0ce73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:55.628655 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3504022-cc27-4fbe-b87c-eec5dbc0ce73 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:55.628994 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3504022-cc27-4fbe-b87c-eec5dbc0ce73 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:55.629142 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3504022-cc27-4fbe-b87c-eec5dbc0ce73 None admin] PATH_INFO: `/v3/services` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:55.629760 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3504022-cc27-4fbe-b87c-eec5dbc0ce73 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:55.631731 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3504022-cc27-4fbe-b87c-eec5dbc0ce73 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:55.638667 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 439/888] 10.222.0.85 () {56 vars in 1087 bytes} [Tue Jun 7 14:20:55 2022] GET /identity/v3/services => generated 1268 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:20:56.076223 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c546489-50e7-41f5-a88c-ccb7ba08019a None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:56.076777 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c546489-50e7-41f5-a88c-ccb7ba08019a None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:56.077162 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c546489-50e7-41f5-a88c-ccb7ba08019a None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:56.078956 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 450/889] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:20:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:20:56.085149 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf631895-5708-4e9a-9d58-90bcbe6fef70 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:56.085588 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf631895-5708-4e9a-9d58-90bcbe6fef70 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:56.085974 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf631895-5708-4e9a-9d58-90bcbe6fef70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:56.104150 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-cf631895-5708-4e9a-9d58-90bcbe6fef70 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:20:56.105530 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-cf631895-5708-4e9a-9d58-90bcbe6fef70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:20:56.192494 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 440/890] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:20:56 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:20:56.215815 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c546489-50e7-41f5-a88c-ccb7ba08019a None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:56.247297 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5465d497-1ac6-4fee-bfb2-27603ea7eac8 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:56.247608 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5465d497-1ac6-4fee-bfb2-27603ea7eac8 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:56.247864 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5465d497-1ac6-4fee-bfb2-27603ea7eac8 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:56.249552 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 441/891] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:20:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:20:56.259814 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c80224-80a5-48e4-828b-6f6f701ced54 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:56.260238 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c80224-80a5-48e4-828b-6f6f701ced54 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:56.260650 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c80224-80a5-48e4-828b-6f6f701ced54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:56.279313 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-30c80224-80a5-48e4-828b-6f6f701ced54 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:20:56.280737 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-30c80224-80a5-48e4-828b-6f6f701ced54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:20:56.286221 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:56.291235 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:56.330508 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c546489-50e7-41f5-a88c-ccb7ba08019a None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:56.353911 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 442/892] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:20:56 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:20:56.380180 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee58dd1-12db-42e9-94f6-c2fd9ca6eaa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:56.382358 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c80224-80a5-48e4-828b-6f6f701ced54 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:56.383778 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee58dd1-12db-42e9-94f6-c2fd9ca6eaa9 None admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:56.384301 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee58dd1-12db-42e9-94f6-c2fd9ca6eaa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:56.384861 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee58dd1-12db-42e9-94f6-c2fd9ca6eaa9 None admin] PATH_INFO: `/v3/services` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:56.386403 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee58dd1-12db-42e9-94f6-c2fd9ca6eaa9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:56.389395 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee58dd1-12db-42e9-94f6-c2fd9ca6eaa9 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:56.477924 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 451/893] 10.222.0.85 () {60 vars in 1141 bytes} [Tue Jun 7 14:20:56 2022] POST /identity/v3/services => generated 241 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:20:56.508779 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c80224-80a5-48e4-828b-6f6f701ced54 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:56.555560 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9eb4ae-df31-4c94-900c-0a10acf43c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:56.556908 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9eb4ae-df31-4c94-900c-0a10acf43c60 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:56.557408 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9eb4ae-df31-4c94-900c-0a10acf43c60 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:56.557956 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9eb4ae-df31-4c94-900c-0a10acf43c60 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:56.570874 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:56.572391 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9eb4ae-df31-4c94-900c-0a10acf43c60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:20:56.581364 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-da9eb4ae-df31-4c94-900c-0a10acf43c60 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 14:20:56.581364 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:56.581364 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:56.581364 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:56.581364 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:56.581364 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:56.581364 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:56.581364 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:56.581364 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:56.581364 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:56.581364 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:56.581364 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:56.581364 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:20:56.581364 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:20:56.581364 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:20:56.581364 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:20:56.582632 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:20:56.582632 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:20:56.582632 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:20:56.582632 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:20:56.582632 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:56.582632 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:56.582632 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:56.582632 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:56.582632 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:56.582632 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:56.582632 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:56.582632 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:56.582632 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:56.582632 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:56.582632 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:56.582632 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:56.583588 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:56.583588 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:56.583588 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:56.583588 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:56.583588 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:20:56.583588 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:20:56.583588 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:20:56.583588 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:20:56.583588 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:20:56.583588 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:20:56.583588 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 14:20:56.583588 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:56.583588 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 443/894] 10.222.0.85 () {56 vars in 1120 bytes} [Tue Jun 7 14:20:56 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:20:56.593535 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee58dd1-12db-42e9-94f6-c2fd9ca6eaa9 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:56.711571 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee58dd1-12db-42e9-94f6-c2fd9ca6eaa9 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:56.717972 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e61c819-934b-4a3c-91e9-a55d5d70b5b2 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:56.718457 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e61c819-934b-4a3c-91e9-a55d5d70b5b2 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:56.719427 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e61c819-934b-4a3c-91e9-a55d5d70b5b2 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:56.721574 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 444/895] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:20:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:20:56.727598 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065bf9bb-edc0-4d91-949a-ae1938357d6c None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:56.728023 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065bf9bb-edc0-4d91-949a-ae1938357d6c None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:56.728204 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065bf9bb-edc0-4d91-949a-ae1938357d6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:56.734990 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:56.749366 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-065bf9bb-edc0-4d91-949a-ae1938357d6c None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:20:56.751367 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-065bf9bb-edc0-4d91-949a-ae1938357d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:20:56.756607 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b9643a-ed47-4c85-b32e-52969015e008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:56.757710 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b9643a-ed47-4c85-b32e-52969015e008 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:56.758198 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b9643a-ed47-4c85-b32e-52969015e008 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:56.758726 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b9643a-ed47-4c85-b32e-52969015e008 None admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:56.759702 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b9643a-ed47-4c85-b32e-52969015e008 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:56.760080 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b9643a-ed47-4c85-b32e-52969015e008 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:56.761579 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b9643a-ed47-4c85-b32e-52969015e008 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:56.777158 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 452/896] 10.222.0.85 () {56 vars in 1115 bytes} [Tue Jun 7 14:20:56 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:56.785705 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b9643a-ed47-4c85-b32e-52969015e008 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:56.852102 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 445/897] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:20:56 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 14:20:56.877901 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065bf9bb-edc0-4d91-949a-ae1938357d6c None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:56.914428 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b9643a-ed47-4c85-b32e-52969015e008 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:56.965763 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7447f192-79de-409a-a636-05cf9ca1aa39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:56.966759 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7447f192-79de-409a-a636-05cf9ca1aa39 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:56.967193 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7447f192-79de-409a-a636-05cf9ca1aa39 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:56.967800 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7447f192-79de-409a-a636-05cf9ca1aa39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:56.981059 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7447f192-79de-409a-a636-05cf9ca1aa39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:56.982758 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7447f192-79de-409a-a636-05cf9ca1aa39 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:57.003397 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-7447f192-79de-409a-a636-05cf9ca1aa39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:57.003397 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:57.003397 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:57.003397 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:57.003397 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:57.003397 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:57.003397 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:57.003397 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:57.003397 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:57.003397 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:57.003397 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:57.003397 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:57.003397 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:20:57.003397 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:20:57.003397 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:20:57.004605 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:20:57.004605 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:57.004605 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:57.004605 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:57.004605 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:57.004605 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:57.004605 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:57.004605 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:57.004605 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:57.004605 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:57.004605 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:57.004605 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:57.004605 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:57.004605 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:57.004605 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:57.004605 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:57.004605 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:57.005557 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:20:57.005557 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:20:57.005557 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:57.005557 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:57.007621 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 453/898] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:20:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:57.015771 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7447f192-79de-409a-a636-05cf9ca1aa39 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:57.015950 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065bf9bb-edc0-4d91-949a-ae1938357d6c None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:57.045408 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:57.052808 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:57.060991 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2129ceb7-aa84-4a96-b26b-c6fdcf5da7e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:57.062110 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2129ceb7-aa84-4a96-b26b-c6fdcf5da7e0 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:57.062838 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2129ceb7-aa84-4a96-b26b-c6fdcf5da7e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:57.063187 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2129ceb7-aa84-4a96-b26b-c6fdcf5da7e0 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:57.064206 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2129ceb7-aa84-4a96-b26b-c6fdcf5da7e0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:57.066331 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2129ceb7-aa84-4a96-b26b-c6fdcf5da7e0 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:57.084263 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-2129ceb7-aa84-4a96-b26b-c6fdcf5da7e0 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 07 14:20:57.084263 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:57.084263 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 14:20:57.084263 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 14:20:57.084263 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 14:20:57.084263 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 14:20:57.084263 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 14:20:57.084263 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:20:57.084263 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 07 14:20:57.084263 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:57.084263 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 14:20:57.084263 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:57.084263 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:57.084263 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:57.084263 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:57.084263 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:57.085312 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:57.085312 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:57.085312 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:57.085312 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:57.085312 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:57.085312 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:57.085312 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:57.085312 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 14:20:57.085312 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 14:20:57.085312 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 14:20:57.085312 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 14:20:57.085312 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:57.085312 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:57.085312 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:57.085312 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:57.085312 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:57.086140 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:57.086140 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:57.086140 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:57.086140 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:57.086140 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:57.086140 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:57.086140 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:57.086140 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:57.086140 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:57.086140 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:57.086140 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:57.086140 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 14:20:57.086140 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:20:57.086140 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 07 14:20:57.086140 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:57.086140 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 446/899] 10.222.0.85 () {56 vars in 1132 bytes} [Tue Jun 7 14:20:56 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:20:57.093065 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2129ceb7-aa84-4a96-b26b-c6fdcf5da7e0 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:57.131564 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7447f192-79de-409a-a636-05cf9ca1aa39 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:57.178771 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486d6ff9-4ebe-42dc-9412-2c0964e344df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:57.179440 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486d6ff9-4ebe-42dc-9412-2c0964e344df None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:57.179726 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486d6ff9-4ebe-42dc-9412-2c0964e344df None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:57.180006 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486d6ff9-4ebe-42dc-9412-2c0964e344df None admin] PATH_INFO: `/v3/domains` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:57.180766 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486d6ff9-4ebe-42dc-9412-2c0964e344df None admin] RBAC: Adding query filter params (name=Default) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:57.181077 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486d6ff9-4ebe-42dc-9412-2c0964e344df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:57.182514 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486d6ff9-4ebe-42dc-9412-2c0964e344df None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:57.200410 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 454/900] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jun 7 14:20:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:57.201385 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486d6ff9-4ebe-42dc-9412-2c0964e344df None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:57.221109 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2129ceb7-aa84-4a96-b26b-c6fdcf5da7e0 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:57.269058 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54005ec-0169-4011-a74f-798aaabd33d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:57.269676 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54005ec-0169-4011-a74f-798aaabd33d8 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:57.270187 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54005ec-0169-4011-a74f-798aaabd33d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:57.270857 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54005ec-0169-4011-a74f-798aaabd33d8 None admin] PATH_INFO: `/v3/services` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:57.272395 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54005ec-0169-4011-a74f-798aaabd33d8 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:57.272721 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54005ec-0169-4011-a74f-798aaabd33d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:57.274097 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54005ec-0169-4011-a74f-798aaabd33d8 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:57.281873 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 447/901] 10.222.0.85 () {56 vars in 1126 bytes} [Tue Jun 7 14:20:57 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:20:57.288511 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54005ec-0169-4011-a74f-798aaabd33d8 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:57.312217 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:57.326896 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486d6ff9-4ebe-42dc-9412-2c0964e344df None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:57.357806 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:57.375570 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28cc366-0b4e-4478-8ae6-c4bffdc584dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:57.376768 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28cc366-0b4e-4478-8ae6-c4bffdc584dc None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:57.377242 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28cc366-0b4e-4478-8ae6-c4bffdc584dc None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:57.377737 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28cc366-0b4e-4478-8ae6-c4bffdc584dc None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:57.388648 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28cc366-0b4e-4478-8ae6-c4bffdc584dc None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:57.390912 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28cc366-0b4e-4478-8ae6-c4bffdc584dc None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:57.406648 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54005ec-0169-4011-a74f-798aaabd33d8 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:57.415196 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-d28cc366-0b4e-4478-8ae6-c4bffdc584dc None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 14:20:57.415196 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:57.415196 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:57.415196 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:57.415196 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:57.415196 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:57.415196 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:57.415196 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:57.415196 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:57.415196 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:57.415196 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:57.415196 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:57.415196 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 14:20:57.415196 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 14:20:57.415196 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 14:20:57.415196 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 14:20:57.417140 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:57.417140 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:57.417140 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 14:20:57.417140 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:20:57.417140 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 14:20:57.417140 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:20:57.417140 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:57.417140 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:57.417140 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:57.417140 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:57.417140 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:57.417140 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:57.417140 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:57.417140 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:57.417140 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:57.417140 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:57.417140 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:57.417140 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:57.418443 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:57.418443 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:57.418443 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 14:20:57.418443 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 14:20:57.418443 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 14:20:57.418443 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 14:20:57.418443 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 14:20:57.418443 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 14:20:57.418443 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 14:20:57.418443 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:57.422303 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 455/902] 10.222.0.85 () {56 vars in 1117 bytes} [Tue Jun 7 14:20:57 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:57.427964 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28cc366-0b4e-4478-8ae6-c4bffdc584dc None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:57.461190 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54967df6-0ec9-46d1-a9c4-aac1dfbffaff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:57.462630 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54967df6-0ec9-46d1-a9c4-aac1dfbffaff None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:57.463077 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54967df6-0ec9-46d1-a9c4-aac1dfbffaff None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:57.463807 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54967df6-0ec9-46d1-a9c4-aac1dfbffaff None admin] PATH_INFO: `/v3/services` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:57.464960 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54967df6-0ec9-46d1-a9c4-aac1dfbffaff None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:57.465630 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54967df6-0ec9-46d1-a9c4-aac1dfbffaff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:57.468590 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54967df6-0ec9-46d1-a9c4-aac1dfbffaff None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:57.484202 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 448/903] 10.222.0.85 () {56 vars in 1126 bytes} [Tue Jun 7 14:20:57 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:20:57.485696 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54967df6-0ec9-46d1-a9c4-aac1dfbffaff None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:57.559127 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28cc366-0b4e-4478-8ae6-c4bffdc584dc None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:57.598175 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30848c9f-a8ef-4d1a-8017-ddf88ae0381b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:57.599340 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30848c9f-a8ef-4d1a-8017-ddf88ae0381b None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:57.599873 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30848c9f-a8ef-4d1a-8017-ddf88ae0381b None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:57.600351 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30848c9f-a8ef-4d1a-8017-ddf88ae0381b None admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:57.601412 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30848c9f-a8ef-4d1a-8017-ddf88ae0381b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:57.601927 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54967df6-0ec9-46d1-a9c4-aac1dfbffaff None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:57.602158 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30848c9f-a8ef-4d1a-8017-ddf88ae0381b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:57.603914 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30848c9f-a8ef-4d1a-8017-ddf88ae0381b None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:57.625923 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:57.631545 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 456/904] 10.222.0.85 () {56 vars in 1149 bytes} [Tue Jun 7 14:20:57 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:57.633544 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30848c9f-a8ef-4d1a-8017-ddf88ae0381b None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:57.638598 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721ea1b5-5003-4d9a-a4a5-6bc8c31e8561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:57.639273 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721ea1b5-5003-4d9a-a4a5-6bc8c31e8561 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:57.639628 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721ea1b5-5003-4d9a-a4a5-6bc8c31e8561 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:57.639915 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721ea1b5-5003-4d9a-a4a5-6bc8c31e8561 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:57.642657 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721ea1b5-5003-4d9a-a4a5-6bc8c31e8561 None admin] RBAC: Adding query filter params (interface=public, service_id=eecfac90a76f440593d713e17e61c521, region_id=RegionOne) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:57.642657 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721ea1b5-5003-4d9a-a4a5-6bc8c31e8561 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:57.642657 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721ea1b5-5003-4d9a-a4a5-6bc8c31e8561 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:57.648272 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:57.649684 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 449/905] 10.222.0.85 () {56 vars in 1251 bytes} [Tue Jun 7 14:20:57 2022] GET /identity/v3/endpoints?service_id=eecfac90a76f440593d713e17e61c521&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:20:57.654973 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721ea1b5-5003-4d9a-a4a5-6bc8c31e8561 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:57.765335 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30848c9f-a8ef-4d1a-8017-ddf88ae0381b None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:57.788200 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721ea1b5-5003-4d9a-a4a5-6bc8c31e8561 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:57.809457 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c5cb48-db90-4119-9a19-4279b1548068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:57.809457 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c5cb48-db90-4119-9a19-4279b1548068 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:57.809457 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c5cb48-db90-4119-9a19-4279b1548068 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:57.810695 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c5cb48-db90-4119-9a19-4279b1548068 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:57.835409 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6189cce8-7764-4eef-b901-0f3a2ebea728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:57.836542 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6189cce8-7764-4eef-b901-0f3a2ebea728 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:57.837558 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6189cce8-7764-4eef-b901-0f3a2ebea728 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:57.838062 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6189cce8-7764-4eef-b901-0f3a2ebea728 None admin] PATH_INFO: `/v3/services` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:57.838970 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6189cce8-7764-4eef-b901-0f3a2ebea728 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:57.840674 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6189cce8-7764-4eef-b901-0f3a2ebea728 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:57.864710 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 450/906] 10.222.0.85 () {56 vars in 1087 bytes} [Tue Jun 7 14:20:57 2022] GET /identity/v3/services => generated 1497 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:20:57.915218 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c5cb48-db90-4119-9a19-4279b1548068 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:57.917559 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c5cb48-db90-4119-9a19-4279b1548068 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:57.933865 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:58.037651 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 457/907] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jun 7 14:20:57 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 14:20:58.039488 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505faf9e-096d-439b-9b7e-33b248ba3604 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:58.039964 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c5cb48-db90-4119-9a19-4279b1548068 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:58.040259 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505faf9e-096d-439b-9b7e-33b248ba3604 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:58.040767 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505faf9e-096d-439b-9b7e-33b248ba3604 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:58.042805 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 451/908] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:20:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:20:58.048070 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ca234a-1039-46eb-a836-17e416649d29 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:58.048396 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ca234a-1039-46eb-a836-17e416649d29 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:58.050824 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ca234a-1039-46eb-a836-17e416649d29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:58.067672 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-a5ca234a-1039-46eb-a836-17e416649d29 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:20:58.068978 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-a5ca234a-1039-46eb-a836-17e416649d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:20:58.128564 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:58.155968 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 452/909] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:20:58 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 14:20:58.171506 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c5cb48-db90-4119-9a19-4279b1548068 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:58.180203 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ca234a-1039-46eb-a836-17e416649d29 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:58.204776 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:58.210303 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234fc05b-15c0-41e6-9e87-0f6288655f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:58.211182 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234fc05b-15c0-41e6-9e87-0f6288655f8d None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:58.211477 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234fc05b-15c0-41e6-9e87-0f6288655f8d None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:58.211760 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234fc05b-15c0-41e6-9e87-0f6288655f8d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:58.223329 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234fc05b-15c0-41e6-9e87-0f6288655f8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:58.225657 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234fc05b-15c0-41e6-9e87-0f6288655f8d None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:58.245442 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-234fc05b-15c0-41e6-9e87-0f6288655f8d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:58.245442 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:58.245442 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:58.245442 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:58.245442 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:58.245442 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:58.245442 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:58.245442 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:58.245442 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:58.245442 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:58.245442 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:58.245442 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:58.245442 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:20:58.245442 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:20:58.245442 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:20:58.246403 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:20:58.246403 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:58.246403 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:58.246403 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:58.246403 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:58.246403 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:58.246403 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:58.246403 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:58.246403 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:58.246403 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:58.246403 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:58.246403 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:58.246403 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:58.246403 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:58.246403 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:58.246403 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:58.246403 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:58.247298 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:20:58.247298 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:20:58.247298 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:20:58.247298 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:58.247298 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 458/910] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:20:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:20:58.255159 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234fc05b-15c0-41e6-9e87-0f6288655f8d None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:58.329609 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ca234a-1039-46eb-a836-17e416649d29 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:58.363539 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234fc05b-15c0-41e6-9e87-0f6288655f8d None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:58.366748 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcc4cc3-c434-4b52-b7c2-993ded9e0765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:58.367556 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc4cc3-c434-4b52-b7c2-993ded9e0765 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:58.367844 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc4cc3-c434-4b52-b7c2-993ded9e0765 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:58.368124 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc4cc3-c434-4b52-b7c2-993ded9e0765 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:58.368814 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcc4cc3-c434-4b52-b7c2-993ded9e0765 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:58.370452 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcc4cc3-c434-4b52-b7c2-993ded9e0765 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:58.384538 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-8bcc4cc3-c434-4b52-b7c2-993ded9e0765 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 07 14:20:58.384538 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:58.384538 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 14:20:58.384538 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 14:20:58.384538 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 14:20:58.384538 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 14:20:58.384538 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 14:20:58.384538 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:20:58.384538 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 07 14:20:58.384538 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:58.384538 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 14:20:58.384538 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:58.384538 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:58.384538 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:58.384538 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:58.384538 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:58.384538 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:58.388299 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:58.388299 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:58.388299 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:58.388299 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:58.388299 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:58.388299 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:58.388299 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 14:20:58.388299 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 14:20:58.388299 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 14:20:58.388299 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 14:20:58.388299 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:58.388299 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:58.388299 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:58.388299 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:58.388299 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:58.388299 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:58.389368 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:58.389368 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:58.389368 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:58.389368 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:58.389368 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:58.389368 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:58.389368 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:58.389368 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:58.389368 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:58.389368 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:58.389368 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 14:20:58.389368 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:20:58.389368 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 07 14:20:58.389368 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:58.389368 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 453/911] 10.222.0.85 () {56 vars in 1114 bytes} [Tue Jun 7 14:20:58 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:20:58.397405 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcc4cc3-c434-4b52-b7c2-993ded9e0765 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:58.407524 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1913a0b-8afc-4e50-8ef0-42d975a130ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:58.408199 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1913a0b-8afc-4e50-8ef0-42d975a130ad None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:58.408551 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1913a0b-8afc-4e50-8ef0-42d975a130ad None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:58.408834 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1913a0b-8afc-4e50-8ef0-42d975a130ad None admin] PATH_INFO: `/v3/domains` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:58.409566 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1913a0b-8afc-4e50-8ef0-42d975a130ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:58.409885 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1913a0b-8afc-4e50-8ef0-42d975a130ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:58.411266 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1913a0b-8afc-4e50-8ef0-42d975a130ad None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:58.428563 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 459/912] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jun 7 14:20:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:20:58.430630 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1913a0b-8afc-4e50-8ef0-42d975a130ad None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:58.457247 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:58.460555 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:58.526984 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcc4cc3-c434-4b52-b7c2-993ded9e0765 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:58.555222 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1913a0b-8afc-4e50-8ef0-42d975a130ad None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:58.565377 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c471acfd-c706-498c-a596-41f3075ce857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:58.566685 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c471acfd-c706-498c-a596-41f3075ce857 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:58.566685 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c471acfd-c706-498c-a596-41f3075ce857 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:58.567033 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c471acfd-c706-498c-a596-41f3075ce857 None admin] PATH_INFO: `/v3/services` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:58.567875 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c471acfd-c706-498c-a596-41f3075ce857 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:58.568936 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c471acfd-c706-498c-a596-41f3075ce857 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:58.570528 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c471acfd-c706-498c-a596-41f3075ce857 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:58.580993 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 454/913] 10.222.0.85 () {56 vars in 1114 bytes} [Tue Jun 7 14:20:58 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:58.582373 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c471acfd-c706-498c-a596-41f3075ce857 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:58.606731 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3053dcac-b07d-4a6c-92a7-e999c10693c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:58.607533 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3053dcac-b07d-4a6c-92a7-e999c10693c8 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:58.607819 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3053dcac-b07d-4a6c-92a7-e999c10693c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:58.608104 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3053dcac-b07d-4a6c-92a7-e999c10693c8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:58.615806 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3053dcac-b07d-4a6c-92a7-e999c10693c8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:58.617294 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3053dcac-b07d-4a6c-92a7-e999c10693c8 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:58.632712 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-3053dcac-b07d-4a6c-92a7-e999c10693c8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:20:58.632712 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:58.632712 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:58.632712 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:58.632712 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:58.632712 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:58.632712 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:58.632712 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:58.632712 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:58.632712 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:58.632712 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:58.632712 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:58.632712 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 14:20:58.632712 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 14:20:58.632712 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 14:20:58.634167 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 14:20:58.634167 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:58.634167 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:58.634167 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:58.634167 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:58.634167 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:58.634167 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:58.634167 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:58.634167 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:58.634167 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:58.634167 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:58.634167 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:58.634167 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:58.634167 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:58.634167 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:58.634167 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:58.634167 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:58.635028 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 14:20:58.635028 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 14:20:58.635028 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 14:20:58.635028 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 14:20:58.635028 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 14:20:58.635028 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 14:20:58.635028 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:20:58.635028 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:20:58.638880 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 460/914] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jun 7 14:20:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:58.639785 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3053dcac-b07d-4a6c-92a7-e999c10693c8 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:58.718146 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c471acfd-c706-498c-a596-41f3075ce857 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:58.730236 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:58.749338 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:58.759077 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25515f72-bed5-46ed-b696-5e923b1c40ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:58.759801 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25515f72-bed5-46ed-b696-5e923b1c40ef None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:58.760148 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25515f72-bed5-46ed-b696-5e923b1c40ef None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:58.760464 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25515f72-bed5-46ed-b696-5e923b1c40ef None admin] PATH_INFO: `/v3/services` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:58.761189 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25515f72-bed5-46ed-b696-5e923b1c40ef None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:58.761506 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25515f72-bed5-46ed-b696-5e923b1c40ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:58.762936 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25515f72-bed5-46ed-b696-5e923b1c40ef None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:58.772458 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 455/915] 10.222.0.85 () {56 vars in 1114 bytes} [Tue Jun 7 14:20:58 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:58.774228 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25515f72-bed5-46ed-b696-5e923b1c40ef None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:58.776484 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3053dcac-b07d-4a6c-92a7-e999c10693c8 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:58.823939 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132f9ce9-5f9c-414c-b4b6-636a750a9143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:58.824650 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132f9ce9-5f9c-414c-b4b6-636a750a9143 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:58.824948 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132f9ce9-5f9c-414c-b4b6-636a750a9143 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:58.825328 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132f9ce9-5f9c-414c-b4b6-636a750a9143 None admin] PATH_INFO: `/v3/projects` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:58.826146 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132f9ce9-5f9c-414c-b4b6-636a750a9143 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:58.826460 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132f9ce9-5f9c-414c-b4b6-636a750a9143 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:58.827906 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132f9ce9-5f9c-414c-b4b6-636a750a9143 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:58.848533 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 461/916] 10.222.0.85 () {56 vars in 1148 bytes} [Tue Jun 7 14:20:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:58.850014 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132f9ce9-5f9c-414c-b4b6-636a750a9143 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:58.914575 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25515f72-bed5-46ed-b696-5e923b1c40ef None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:58.958459 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807ea6cb-9654-456d-b49f-c95b70d59fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:58.959168 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807ea6cb-9654-456d-b49f-c95b70d59fb6 None admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:58.959455 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807ea6cb-9654-456d-b49f-c95b70d59fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:58.959816 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807ea6cb-9654-456d-b49f-c95b70d59fb6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:58.960949 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807ea6cb-9654-456d-b49f-c95b70d59fb6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:58.962759 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807ea6cb-9654-456d-b49f-c95b70d59fb6 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:58.971746 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132f9ce9-5f9c-414c-b4b6-636a750a9143 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:59.006213 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b807acba-c99f-4d2b-b8a6-752057b08079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:59.006839 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b807acba-c99f-4d2b-b8a6-752057b08079 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:59.007121 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b807acba-c99f-4d2b-b8a6-752057b08079 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:59.007396 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b807acba-c99f-4d2b-b8a6-752057b08079 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:59.008216 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b807acba-c99f-4d2b-b8a6-752057b08079 None admin] RBAC: Adding query filter params (role.id=5ca16aa9be5740d2902e27a98242edf4, scope.project.id=724785eb74e44daf8736fd01fc0d1839, user.id=ad9d021b171244d796c320aa9f8450a9) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:59.008557 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b807acba-c99f-4d2b-b8a6-752057b08079 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:59.009979 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b807acba-c99f-4d2b-b8a6-752057b08079 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:59.015365 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 462/917] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jun 7 14:20:58 2022] GET /identity/v3/role_assignments?user.id=ad9d021b171244d796c320aa9f8450a9&scope.project.id=724785eb74e44daf8736fd01fc0d1839&role.id=5ca16aa9be5740d2902e27a98242edf4 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:59.016587 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbd96ef-2f45-44ee-9ff8-dbb45bcbae65 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:59.016876 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbd96ef-2f45-44ee-9ff8-dbb45bcbae65 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:59.017150 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbd96ef-2f45-44ee-9ff8-dbb45bcbae65 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:59.020529 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 463/918] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:20:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:20:59.024153 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103ade68-29eb-4041-a544-35f5a679993a None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:59.024476 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103ade68-29eb-4041-a544-35f5a679993a None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:59.024756 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103ade68-29eb-4041-a544-35f5a679993a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:59.038434 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:59.045074 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-103ade68-29eb-4041-a544-35f5a679993a None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:20:59.046287 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-103ade68-29eb-4041-a544-35f5a679993a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:20:59.066523 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 456/919] 10.222.0.85 () {60 vars in 1144 bytes} [Tue Jun 7 14:20:58 2022] POST /identity/v3/endpoints => generated 346 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:20:59.142159 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 464/920] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:20:59 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 14:20:59.166505 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807ea6cb-9654-456d-b49f-c95b70d59fb6 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:59.203073 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:59.292306 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807ea6cb-9654-456d-b49f-c95b70d59fb6 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:59.327959 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b527a579-fbd0-4e0d-8e1b-0e04335a20d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:59.329404 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b527a579-fbd0-4e0d-8e1b-0e04335a20d6 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:59.329956 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b527a579-fbd0-4e0d-8e1b-0e04335a20d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:59.330233 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b527a579-fbd0-4e0d-8e1b-0e04335a20d6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:59.330901 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b527a579-fbd0-4e0d-8e1b-0e04335a20d6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:59.332889 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b527a579-fbd0-4e0d-8e1b-0e04335a20d6 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:59.345288 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-b527a579-fbd0-4e0d-8e1b-0e04335a20d6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 14:20:59.345288 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:59.345288 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 14:20:59.345288 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 14:20:59.345288 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 14:20:59.345288 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 14:20:59.345288 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 14:20:59.345288 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:20:59.345288 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 14:20:59.345288 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:59.345288 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 14:20:59.345288 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:59.345288 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:20:59.345288 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:20:59.345288 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:20:59.345288 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:20:59.345288 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:20:59.346307 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:20:59.346307 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:20:59.346307 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:20:59.346307 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:20:59.346307 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:20:59.346307 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:20:59.346307 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 14:20:59.346307 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 14:20:59.346307 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 14:20:59.346307 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 14:20:59.346307 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:20:59.346307 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:20:59.346307 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:20:59.346307 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:20:59.346307 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:20:59.346307 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:20:59.347135 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:20:59.347135 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:20:59.347135 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:20:59.347135 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:20:59.347135 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:20:59.347135 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:20:59.347135 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:20:59.347135 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:20:59.347135 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:20:59.347135 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:20:59.347135 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 14:20:59.347135 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:20:59.347135 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 14:20:59.347135 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:20:59.347135 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 457/921] 10.222.0.85 () {56 vars in 1126 bytes} [Tue Jun 7 14:20:59 2022] GET /identity/v3/services/object-store => generated 93 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:20:59.353336 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103ade68-29eb-4041-a544-35f5a679993a None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:59.459483 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103ade68-29eb-4041-a544-35f5a679993a None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:59.490327 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:59.495907 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fa752c-9869-4a2c-8fb6-e7fc7e79da0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:59.496598 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fa752c-9869-4a2c-8fb6-e7fc7e79da0a None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:59.496883 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fa752c-9869-4a2c-8fb6-e7fc7e79da0a None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:59.497158 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fa752c-9869-4a2c-8fb6-e7fc7e79da0a None admin] PATH_INFO: `/v3/services` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:59.497864 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fa752c-9869-4a2c-8fb6-e7fc7e79da0a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:59.498240 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fa752c-9869-4a2c-8fb6-e7fc7e79da0a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:59.499668 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fa752c-9869-4a2c-8fb6-e7fc7e79da0a None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:59.505065 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 465/922] 10.222.0.85 () {56 vars in 1122 bytes} [Tue Jun 7 14:20:59 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:59.510835 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b527a579-fbd0-4e0d-8e1b-0e04335a20d6 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:59.611291 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:20:59.621133 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b527a579-fbd0-4e0d-8e1b-0e04335a20d6 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:59.658667 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8815d5-1926-49df-a9cc-4d86bcdde686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:59.659341 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8815d5-1926-49df-a9cc-4d86bcdde686 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:59.659629 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8815d5-1926-49df-a9cc-4d86bcdde686 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:59.659909 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8815d5-1926-49df-a9cc-4d86bcdde686 None admin] PATH_INFO: `/v3/services` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:59.660656 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8815d5-1926-49df-a9cc-4d86bcdde686 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:59.660968 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8815d5-1926-49df-a9cc-4d86bcdde686 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:59.662900 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8815d5-1926-49df-a9cc-4d86bcdde686 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:59.668630 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 458/923] 10.222.0.85 () {56 vars in 1122 bytes} [Tue Jun 7 14:20:59 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:59.674375 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fa752c-9869-4a2c-8fb6-e7fc7e79da0a None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:59.778424 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fa752c-9869-4a2c-8fb6-e7fc7e79da0a None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:59.813264 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e572b78b-87a1-4780-bfe0-25dc521acb7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:59.813928 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e572b78b-87a1-4780-bfe0-25dc521acb7f None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:59.814203 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e572b78b-87a1-4780-bfe0-25dc521acb7f None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:59.814475 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e572b78b-87a1-4780-bfe0-25dc521acb7f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:59.815267 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e572b78b-87a1-4780-bfe0-25dc521acb7f None admin] RBAC: Adding query filter params (interface=public, service_id=b5b88f38fd6a4ac9b78bde30a831454e, region_id=RegionOne) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:20:59.815574 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e572b78b-87a1-4780-bfe0-25dc521acb7f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:59.816962 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e572b78b-87a1-4780-bfe0-25dc521acb7f None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:59.823106 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 466/924] 10.222.0.85 () {56 vars in 1251 bytes} [Tue Jun 7 14:20:59 2022] GET /identity/v3/endpoints?service_id=b5b88f38fd6a4ac9b78bde30a831454e&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:20:59.828647 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8815d5-1926-49df-a9cc-4d86bcdde686 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:20:59.948953 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8815d5-1926-49df-a9cc-4d86bcdde686 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:20:59.987845 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d630561-88f1-49fa-aae7-d8db524f7680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:20:59.988545 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d630561-88f1-49fa-aae7-d8db524f7680 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:20:59.988839 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d630561-88f1-49fa-aae7-d8db524f7680 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:20:59.989122 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d630561-88f1-49fa-aae7-d8db524f7680 None admin] PATH_INFO: `/v3/services` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:20:59.989812 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d630561-88f1-49fa-aae7-d8db524f7680 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:20:59.991276 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d630561-88f1-49fa-aae7-d8db524f7680 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:20:59.998586 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 459/925] 10.222.0.85 () {56 vars in 1087 bytes} [Tue Jun 7 14:20:59 2022] GET /identity/v3/services => generated 1497 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:21:00.280911 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529c5822-cd5d-4c71-8a4f-dea405ce3cdb None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:00.281261 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529c5822-cd5d-4c71-8a4f-dea405ce3cdb None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:00.281476 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529c5822-cd5d-4c71-8a4f-dea405ce3cdb None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:00.282986 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 467/926] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:21:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:21:00.289158 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb4edf0-006e-4a61-9e91-6a574fa1e9dc None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:00.289441 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb4edf0-006e-4a61-9e91-6a574fa1e9dc None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:00.289722 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb4edf0-006e-4a61-9e91-6a574fa1e9dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:00.307000 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-5eb4edf0-006e-4a61-9e91-6a574fa1e9dc None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:00.308564 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-5eb4edf0-006e-4a61-9e91-6a574fa1e9dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:00.323936 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:00.386780 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 460/927] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:21:00 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 14:21:00.413422 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529c5822-cd5d-4c71-8a4f-dea405ce3cdb None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:00.475579 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:00.518855 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529c5822-cd5d-4c71-8a4f-dea405ce3cdb None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:00.556339 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c3f38d-4efd-4c94-97e9-be6805fdc3f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:00.557511 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c3f38d-4efd-4c94-97e9-be6805fdc3f8 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:00.558003 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c3f38d-4efd-4c94-97e9-be6805fdc3f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:00.558526 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c3f38d-4efd-4c94-97e9-be6805fdc3f8 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:00.559384 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c3f38d-4efd-4c94-97e9-be6805fdc3f8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:00.560984 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c3f38d-4efd-4c94-97e9-be6805fdc3f8 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:00.573298 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-41c3f38d-4efd-4c94-97e9-be6805fdc3f8 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 07 14:21:00.573298 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:21:00.573298 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 14:21:00.573298 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 14:21:00.573298 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 14:21:00.573298 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 14:21:00.573298 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 14:21:00.573298 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:21:00.573298 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 07 14:21:00.573298 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:21:00.573298 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 14:21:00.573298 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:21:00.573298 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:21:00.573298 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:21:00.573298 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:21:00.573298 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:21:00.574219 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:21:00.574219 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:21:00.574219 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:21:00.574219 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:21:00.574219 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:21:00.574219 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:21:00.574219 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:21:00.574219 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 14:21:00.574219 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 14:21:00.574219 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 14:21:00.574219 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 14:21:00.574219 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:21:00.574219 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:21:00.574219 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:21:00.574219 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:21:00.574219 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:21:00.575071 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:21:00.575071 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:21:00.575071 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:21:00.575071 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:21:00.575071 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:21:00.575071 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:21:00.575071 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:21:00.575071 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:21:00.575071 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:21:00.575071 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:21:00.575071 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:21:00.575071 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 14:21:00.575071 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:21:00.575071 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 07 14:21:00.575071 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:21:00.582601 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 468/928] 10.222.0.85 () {56 vars in 1132 bytes} [Tue Jun 7 14:21:00 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:21:00.584320 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb4edf0-006e-4a61-9e91-6a574fa1e9dc None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:00.698797 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb4edf0-006e-4a61-9e91-6a574fa1e9dc None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:00.734555 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2457c58-6943-404c-8074-75d6efc65e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:00.735623 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2457c58-6943-404c-8074-75d6efc65e42 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:00.736136 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2457c58-6943-404c-8074-75d6efc65e42 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:00.736662 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2457c58-6943-404c-8074-75d6efc65e42 None admin] PATH_INFO: `/v3/services` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:00.737582 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2457c58-6943-404c-8074-75d6efc65e42 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:00.738096 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2457c58-6943-404c-8074-75d6efc65e42 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:00.739764 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2457c58-6943-404c-8074-75d6efc65e42 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:00.750589 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 461/929] 10.222.0.85 () {56 vars in 1126 bytes} [Tue Jun 7 14:21:00 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:21:00.752245 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c3f38d-4efd-4c94-97e9-be6805fdc3f8 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:00.859789 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c3f38d-4efd-4c94-97e9-be6805fdc3f8 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:00.885544 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:00.896933 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b78d931-33cd-49b8-a2a4-d5f6048f5b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:00.897611 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b78d931-33cd-49b8-a2a4-d5f6048f5b8a None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:00.900139 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b78d931-33cd-49b8-a2a4-d5f6048f5b8a None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:00.900139 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b78d931-33cd-49b8-a2a4-d5f6048f5b8a None admin] PATH_INFO: `/v3/services` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:00.900139 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b78d931-33cd-49b8-a2a4-d5f6048f5b8a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:00.900139 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b78d931-33cd-49b8-a2a4-d5f6048f5b8a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:00.900571 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b78d931-33cd-49b8-a2a4-d5f6048f5b8a None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:00.905970 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 469/930] 10.222.0.85 () {56 vars in 1126 bytes} [Tue Jun 7 14:21:00 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:21:00.911513 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2457c58-6943-404c-8074-75d6efc65e42 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:00.932407 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:01.026947 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2457c58-6943-404c-8074-75d6efc65e42 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:01.063401 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0f4776-ec0f-42f9-83ce-30b9d5cc70a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:01.064072 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0f4776-ec0f-42f9-83ce-30b9d5cc70a9 None admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:01.064361 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0f4776-ec0f-42f9-83ce-30b9d5cc70a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:01.064670 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0f4776-ec0f-42f9-83ce-30b9d5cc70a9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:01.065432 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0f4776-ec0f-42f9-83ce-30b9d5cc70a9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:01.066799 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0f4776-ec0f-42f9-83ce-30b9d5cc70a9 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:01.185406 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 462/931] 10.222.0.85 () {60 vars in 1144 bytes} [Tue Jun 7 14:21:00 2022] POST /identity/v3/endpoints => generated 347 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:21:01.476650 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96479be-5bd3-40c3-aa61-c671b40bf3e9 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:01.477170 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96479be-5bd3-40c3-aa61-c671b40bf3e9 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:01.477654 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96479be-5bd3-40c3-aa61-c671b40bf3e9 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:01.480550 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 470/932] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:21:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:21:01.487820 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e62166-3722-4da0-bcbb-0033ea5b174f None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:01.488107 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e62166-3722-4da0-bcbb-0033ea5b174f None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:01.488403 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e62166-3722-4da0-bcbb-0033ea5b174f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:01.505239 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-13e62166-3722-4da0-bcbb-0033ea5b174f None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:01.506933 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-13e62166-3722-4da0-bcbb-0033ea5b174f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:01.526874 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb833e7-8899-402e-9c11-93dbe0893f4e None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:01.527192 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb833e7-8899-402e-9c11-93dbe0893f4e None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:01.527438 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb833e7-8899-402e-9c11-93dbe0893f4e None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:01.529107 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 471/933] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:21:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 14:21:01.534052 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41959991-b468-4626-978b-3eaf6b386c54 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:01.534349 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41959991-b468-4626-978b-3eaf6b386c54 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:01.534635 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41959991-b468-4626-978b-3eaf6b386c54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:01.551952 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-41959991-b468-4626-978b-3eaf6b386c54 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:01.553211 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-41959991-b468-4626-978b-3eaf6b386c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:01.586135 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:01.600557 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 463/934] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:21:01 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:21:01.633796 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e62166-3722-4da0-bcbb-0033ea5b174f None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:01.638085 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 472/935] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:21:01 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 14:21:01.662895 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41959991-b468-4626-978b-3eaf6b386c54 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:01.761525 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e62166-3722-4da0-bcbb-0033ea5b174f None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:01.783208 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41959991-b468-4626-978b-3eaf6b386c54 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:01.801305 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428579b3-ab5d-4e7d-bec6-1835f62d63c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:01.802122 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428579b3-ab5d-4e7d-bec6-1835f62d63c6 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:01.802403 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428579b3-ab5d-4e7d-bec6-1835f62d63c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:01.803212 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428579b3-ab5d-4e7d-bec6-1835f62d63c6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:01.805350 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:01.815757 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428579b3-ab5d-4e7d-bec6-1835f62d63c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:21:01.824955 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f725ed4-0c08-41b6-baa9-c9a6ed2aabde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:01.825706 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f725ed4-0c08-41b6-baa9-c9a6ed2aabde None admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:01.825963 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-428579b3-ab5d-4e7d-bec6-1835f62d63c6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 14:21:01.825963 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:21:01.825963 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:21:01.825963 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:21:01.825963 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:21:01.825963 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:21:01.825963 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:21:01.825963 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:21:01.825963 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:21:01.825963 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:21:01.825963 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:21:01.825963 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:21:01.825963 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:21:01.825963 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:21:01.825963 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:21:01.825963 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:21:01.827062 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:21:01.827062 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:21:01.827062 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:21:01.827062 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:21:01.827062 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:21:01.827062 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:21:01.827062 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:21:01.827062 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:21:01.827062 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:21:01.827062 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:21:01.827062 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:21:01.827062 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:21:01.827062 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:21:01.827062 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:21:01.827062 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:21:01.827062 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:21:01.828264 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:21:01.828264 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:21:01.828264 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:21:01.828264 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:21:01.828264 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:21:01.828264 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:21:01.828264 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:21:01.828264 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:21:01.828264 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:21:01.828264 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:21:01.828264 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 14:21:01.828264 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:21:01.829128 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f725ed4-0c08-41b6-baa9-c9a6ed2aabde None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:01.829128 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f725ed4-0c08-41b6-baa9-c9a6ed2aabde None admin] PATH_INFO: `/v3/projects` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:01.829128 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f725ed4-0c08-41b6-baa9-c9a6ed2aabde None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:01.835539 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f725ed4-0c08-41b6-baa9-c9a6ed2aabde None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:01.835539 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: WARNING keystone.server.flask.common [None req-0f725ed4-0c08-41b6-baa9-c9a6ed2aabde None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 07 14:21:01.836207 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 464/936] 10.222.0.85 () {56 vars in 1120 bytes} [Tue Jun 7 14:21:01 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:21:01.837791 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428579b3-ab5d-4e7d-bec6-1835f62d63c6 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:01.893075 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.sql.core [None req-0f725ed4-0c08-41b6-baa9-c9a6ed2aabde None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 07 14:21:01.893075 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 07 14:21:01.893075 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [parameters: {'id': '143141924c2a4d9eaa31371e381b465d', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 07 14:21:01.893075 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80387) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 07 14:21:01.912832 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:01.917054 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-0f725ed4-0c08-41b6-baa9-c9a6ed2aabde None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 143141924c2a4d9eaa31371e381b465d.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 143141924c2a4d9eaa31371e381b465d. Jun 07 14:21:01.917054 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:21:01.917054 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 07 14:21:01.917054 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 07 14:21:01.917054 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 07 14:21:01.917054 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 07 14:21:01.917054 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 07 14:21:01.917054 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application result = self._query(query) Jun 07 14:21:01.917054 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 07 14:21:01.917054 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application conn.query(q) Jun 07 14:21:01.917054 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 07 14:21:01.917054 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 07 14:21:01.917054 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 07 14:21:01.917976 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application result.read() Jun 07 14:21:01.917976 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 07 14:21:01.917976 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 07 14:21:01.917976 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 07 14:21:01.917976 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 07 14:21:01.917976 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 07 14:21:01.917976 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 07 14:21:01.917976 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 07 14:21:01.917976 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 07 14:21:01.917976 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 07 14:21:01.917976 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:21:01.917976 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 07 14:21:01.917976 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:21:01.917976 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:21:01.917976 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 07 14:21:01.917976 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 07 14:21:01.917976 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 07 14:21:01.917976 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 07 14:21:01.918983 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 07 14:21:01.918983 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application next(self.gen) Jun 07 14:21:01.918983 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 07 14:21:01.918983 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application yield resource Jun 07 14:21:01.918983 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 07 14:21:01.918983 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application next(self.gen) Jun 07 14:21:01.918983 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 07 14:21:01.918983 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application self.session.rollback() Jun 07 14:21:01.918983 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 07 14:21:01.918983 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application self.force_reraise() Jun 07 14:21:01.918983 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 07 14:21:01.918983 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise self.value Jun 07 14:21:01.918983 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 07 14:21:01.918983 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 07 14:21:01.918983 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 07 14:21:01.918983 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application session.commit() Jun 07 14:21:01.918983 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 07 14:21:01.919886 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 07 14:21:01.919886 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 07 14:21:01.919886 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application self._prepare_impl() Jun 07 14:21:01.919886 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 07 14:21:01.919886 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application self.session.flush() Jun 07 14:21:01.919886 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 07 14:21:01.919886 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application self._flush(objects) Jun 07 14:21:01.919886 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 07 14:21:01.919886 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 07 14:21:01.919886 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 07 14:21:01.919886 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application compat.raise_( Jun 07 14:21:01.919886 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 07 14:21:01.919886 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception Jun 07 14:21:01.919886 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 07 14:21:01.919886 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application flush_context.execute() Jun 07 14:21:01.919886 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 07 14:21:01.919886 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rec.execute(self) Jun 07 14:21:01.928807 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 07 14:21:01.928807 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 07 14:21:01.928807 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 07 14:21:01.928807 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 07 14:21:01.928807 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 07 14:21:01.928807 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 07 14:21:01.928807 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 07 14:21:01.928807 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 07 14:21:01.928807 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 07 14:21:01.928807 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 07 14:21:01.928807 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 07 14:21:01.928807 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 07 14:21:01.928807 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 07 14:21:01.928807 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 07 14:21:01.928807 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 07 14:21:01.930413 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 07 14:21:01.930413 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 07 14:21:01.930413 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception Jun 07 14:21:01.930413 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 07 14:21:01.930413 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 07 14:21:01.930413 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 07 14:21:01.930413 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 07 14:21:01.930413 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 07 14:21:01.930413 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application result = self._query(query) Jun 07 14:21:01.930413 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 07 14:21:01.930413 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application conn.query(q) Jun 07 14:21:01.930413 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 07 14:21:01.930413 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 07 14:21:01.930413 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 07 14:21:01.930413 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application result.read() Jun 07 14:21:01.930413 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 07 14:21:01.930413 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 07 14:21:01.932195 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 07 14:21:01.932195 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 07 14:21:01.932195 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 07 14:21:01.932195 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 07 14:21:01.932195 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 07 14:21:01.932195 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 07 14:21:01.932195 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 07 14:21:01.932195 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 07 14:21:01.932195 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [parameters: {'id': '143141924c2a4d9eaa31371e381b465d', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 07 14:21:01.932195 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 07 14:21:01.932195 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:21:01.932195 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 14:21:01.932195 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:21:01.932195 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:21:01.932195 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 07 14:21:01.933897 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 07 14:21:01.933897 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 07 14:21:01.933897 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 07 14:21:01.933897 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 07 14:21:01.933897 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:21:01.933897 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 14:21:01.933897 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:21:01.933897 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:21:01.933897 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:21:01.933897 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:21:01.933897 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:21:01.933897 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:21:01.933897 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:21:01.933897 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:21:01.933897 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:21:01.933897 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:21:01.933897 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:21:01.935284 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:21:01.935284 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 07 14:21:01.935284 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 07 14:21:01.935284 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:21:01.935284 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:21:01.935284 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 07 14:21:01.935284 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 07 14:21:01.935284 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 143141924c2a4d9eaa31371e381b465d. Jun 07 14:21:01.935284 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:21:01.935284 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 473/937] 10.222.0.85 () {60 vars in 1140 bytes} [Tue Jun 7 14:21:01 2022] POST /identity/v3/projects => generated 262 bytes in 261 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:21:01.944450 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f725ed4-0c08-41b6-baa9-c9a6ed2aabde None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:01.965454 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428579b3-ab5d-4e7d-bec6-1835f62d63c6 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:02.013312 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb55a742-20f8-498c-aa77-1409db5ed803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:02.013909 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb55a742-20f8-498c-aa77-1409db5ed803 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:02.014316 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb55a742-20f8-498c-aa77-1409db5ed803 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:02.014592 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb55a742-20f8-498c-aa77-1409db5ed803 None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:02.015351 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb55a742-20f8-498c-aa77-1409db5ed803 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:02.015659 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb55a742-20f8-498c-aa77-1409db5ed803 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:02.017613 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb55a742-20f8-498c-aa77-1409db5ed803 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:02.034243 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 465/938] 10.222.0.85 () {56 vars in 1115 bytes} [Tue Jun 7 14:21:01 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:21:02.037653 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb55a742-20f8-498c-aa77-1409db5ed803 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:02.109497 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f725ed4-0c08-41b6-baa9-c9a6ed2aabde None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:02.164550 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ca818d-937d-46bf-a867-2b24f20b3b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:02.165355 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ca818d-937d-46bf-a867-2b24f20b3b87 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:02.165632 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ca818d-937d-46bf-a867-2b24f20b3b87 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:02.165906 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ca818d-937d-46bf-a867-2b24f20b3b87 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:02.176522 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ca818d-937d-46bf-a867-2b24f20b3b87 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:02.179800 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ca818d-937d-46bf-a867-2b24f20b3b87 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:02.198124 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb55a742-20f8-498c-aa77-1409db5ed803 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:02.198964 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-b6ca818d-937d-46bf-a867-2b24f20b3b87 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:21:02.198964 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:21:02.198964 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:21:02.198964 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:21:02.198964 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:21:02.198964 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:21:02.198964 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:21:02.198964 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:21:02.198964 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:21:02.198964 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:21:02.198964 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:21:02.198964 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:21:02.198964 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 14:21:02.198964 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 14:21:02.198964 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 14:21:02.207619 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 14:21:02.207619 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:21:02.207619 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:21:02.207619 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:21:02.207619 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:21:02.207619 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:21:02.207619 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:21:02.207619 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:21:02.207619 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:21:02.207619 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:21:02.207619 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:21:02.207619 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:21:02.207619 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:21:02.207619 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:21:02.207619 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:21:02.207619 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:21:02.207619 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:21:02.210340 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 14:21:02.210340 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 14:21:02.210340 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 14:21:02.210340 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 14:21:02.210340 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 14:21:02.210340 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 14:21:02.210340 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:21:02.210340 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:21:02.210340 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 474/939] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jun 7 14:21:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:21:02.210340 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ca818d-937d-46bf-a867-2b24f20b3b87 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:02.229609 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:02.250528 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9584aad-b564-4044-97b6-2c21341afb88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:02.251371 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9584aad-b564-4044-97b6-2c21341afb88 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:02.251695 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9584aad-b564-4044-97b6-2c21341afb88 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:02.251978 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9584aad-b564-4044-97b6-2c21341afb88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:02.257403 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:02.266076 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9584aad-b564-4044-97b6-2c21341afb88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:02.268696 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9584aad-b564-4044-97b6-2c21341afb88 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:02.288708 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-b9584aad-b564-4044-97b6-2c21341afb88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:21:02.288708 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:21:02.288708 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:21:02.288708 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:21:02.288708 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:21:02.288708 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:21:02.288708 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:21:02.288708 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:21:02.288708 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:21:02.288708 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:21:02.288708 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:21:02.288708 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:21:02.288708 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:21:02.288708 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:21:02.288708 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:21:02.290441 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:21:02.290441 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:21:02.290441 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:21:02.290441 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:21:02.290441 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:21:02.290441 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:21:02.290441 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:21:02.290441 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:21:02.290441 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:21:02.290441 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:21:02.290441 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:21:02.290441 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:21:02.290441 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:21:02.290441 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:21:02.290441 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:21:02.290441 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:21:02.290441 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:21:02.292204 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:21:02.292204 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:21:02.292204 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:21:02.292204 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:21:02.292204 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 466/940] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:21:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:21:02.305865 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9584aad-b564-4044-97b6-2c21341afb88 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:02.361720 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ca818d-937d-46bf-a867-2b24f20b3b87 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:02.405619 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb1ec36-da9d-4ee5-9a0f-57e0ba38d74e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:02.407043 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb1ec36-da9d-4ee5-9a0f-57e0ba38d74e None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:02.407211 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb1ec36-da9d-4ee5-9a0f-57e0ba38d74e None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:02.407479 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb1ec36-da9d-4ee5-9a0f-57e0ba38d74e None admin] PATH_INFO: `/v3/projects` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:02.408266 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb1ec36-da9d-4ee5-9a0f-57e0ba38d74e None admin] RBAC: Adding query filter params (name=service) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:02.408704 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb1ec36-da9d-4ee5-9a0f-57e0ba38d74e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:02.410252 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb1ec36-da9d-4ee5-9a0f-57e0ba38d74e None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:02.439844 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 475/941] 10.222.0.85 () {56 vars in 1112 bytes} [Tue Jun 7 14:21:02 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:21:02.441305 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9584aad-b564-4044-97b6-2c21341afb88 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:02.441428 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283e9b48-0d55-4cd6-8a65-bebbeee71707 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:02.441597 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283e9b48-0d55-4cd6-8a65-bebbeee71707 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:02.441881 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283e9b48-0d55-4cd6-8a65-bebbeee71707 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:02.443539 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 476/942] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:21:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:21:02.451584 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97926a6c-e30e-447e-a743-d12fa2e4bdd4 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:02.452178 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97926a6c-e30e-447e-a743-d12fa2e4bdd4 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:02.452794 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97926a6c-e30e-447e-a743-d12fa2e4bdd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:02.472912 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-97926a6c-e30e-447e-a743-d12fa2e4bdd4 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:02.475358 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-97926a6c-e30e-447e-a743-d12fa2e4bdd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:02.497423 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa487d4c-252e-424a-9e5a-ccead8e138c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:02.498856 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa487d4c-252e-424a-9e5a-ccead8e138c3 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:02.499314 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa487d4c-252e-424a-9e5a-ccead8e138c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:02.499747 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa487d4c-252e-424a-9e5a-ccead8e138c3 None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:02.500696 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa487d4c-252e-424a-9e5a-ccead8e138c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:02.501198 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa487d4c-252e-424a-9e5a-ccead8e138c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:02.504131 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa487d4c-252e-424a-9e5a-ccead8e138c3 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:02.529679 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 467/943] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jun 7 14:21:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:21:02.532842 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa487d4c-252e-424a-9e5a-ccead8e138c3 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:02.562474 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 477/944] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:21:02 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 14:21:02.577577 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:02.588807 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97926a6c-e30e-447e-a743-d12fa2e4bdd4 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:02.624767 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:02.690530 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa487d4c-252e-424a-9e5a-ccead8e138c3 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:02.736180 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97926a6c-e30e-447e-a743-d12fa2e4bdd4 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:02.741608 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af08a94-0263-4d51-896d-b4554b0a8646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:02.742442 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af08a94-0263-4d51-896d-b4554b0a8646 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:02.742725 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af08a94-0263-4d51-896d-b4554b0a8646 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:02.743006 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af08a94-0263-4d51-896d-b4554b0a8646 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:02.752737 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af08a94-0263-4d51-896d-b4554b0a8646 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:02.754195 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af08a94-0263-4d51-896d-b4554b0a8646 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:02.772787 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-0af08a94-0263-4d51-896d-b4554b0a8646 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 14:21:02.772787 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:21:02.772787 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:21:02.772787 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:21:02.772787 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:21:02.772787 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:21:02.772787 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:21:02.772787 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:21:02.772787 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:21:02.772787 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:21:02.772787 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:21:02.772787 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:21:02.772787 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 14:21:02.772787 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 14:21:02.772787 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 14:21:02.772787 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 14:21:02.773753 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:21:02.773753 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:21:02.773753 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 14:21:02.773753 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:21:02.773753 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 14:21:02.773753 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:21:02.773753 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:21:02.773753 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:21:02.773753 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:21:02.773753 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:21:02.773753 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:21:02.773753 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:21:02.773753 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:21:02.773753 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:21:02.773753 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:21:02.773753 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:21:02.773753 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:21:02.773753 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:21:02.774862 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:21:02.774862 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:21:02.774862 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 14:21:02.774862 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 14:21:02.774862 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 14:21:02.774862 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 14:21:02.774862 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 14:21:02.774862 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 14:21:02.774862 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 14:21:02.774862 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:21:02.776219 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 468/945] 10.222.0.85 () {56 vars in 1117 bytes} [Tue Jun 7 14:21:02 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:21:02.779755 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af08a94-0263-4d51-896d-b4554b0a8646 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:02.784360 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36bc96b-7897-4bb0-ad5a-f039d4dfec9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:02.784682 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36bc96b-7897-4bb0-ad5a-f039d4dfec9c None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:02.784965 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36bc96b-7897-4bb0-ad5a-f039d4dfec9c None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:02.785241 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36bc96b-7897-4bb0-ad5a-f039d4dfec9c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:02.785901 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36bc96b-7897-4bb0-ad5a-f039d4dfec9c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:02.787312 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36bc96b-7897-4bb0-ad5a-f039d4dfec9c None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:02.799947 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-b36bc96b-7897-4bb0-ad5a-f039d4dfec9c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 14:21:02.799947 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:21:02.799947 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 14:21:02.799947 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 14:21:02.799947 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 14:21:02.799947 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 14:21:02.799947 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 14:21:02.799947 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:21:02.799947 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 14:21:02.799947 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:21:02.799947 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 14:21:02.799947 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:21:02.799947 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:21:02.799947 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:21:02.799947 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:21:02.799947 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:21:02.799947 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:21:02.800983 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:21:02.800983 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:21:02.800983 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:21:02.800983 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:21:02.800983 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:21:02.800983 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:21:02.800983 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 14:21:02.800983 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 14:21:02.800983 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 14:21:02.800983 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 14:21:02.800983 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:21:02.800983 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:21:02.800983 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:21:02.800983 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:21:02.800983 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:21:02.800983 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:21:02.801802 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:21:02.801802 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:21:02.801802 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:21:02.801802 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:21:02.801802 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:21:02.801802 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:21:02.801802 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:21:02.801802 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:21:02.801802 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:21:02.801802 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:21:02.801802 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 14:21:02.801802 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:21:02.801802 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 14:21:02.801802 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:21:02.801802 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 478/946] 10.222.0.85 () {56 vars in 1126 bytes} [Tue Jun 7 14:21:02 2022] GET /identity/v3/services/object-store => generated 93 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:21:02.808884 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36bc96b-7897-4bb0-ad5a-f039d4dfec9c None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:02.896121 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:02.910378 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af08a94-0263-4d51-896d-b4554b0a8646 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:02.932999 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:02.960078 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36bc96b-7897-4bb0-ad5a-f039d4dfec9c None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:02.976425 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964690cd-746f-40d9-867b-0913139c5e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:02.977617 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964690cd-746f-40d9-867b-0913139c5e63 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:02.978100 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964690cd-746f-40d9-867b-0913139c5e63 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:02.978547 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964690cd-746f-40d9-867b-0913139c5e63 None admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:02.980634 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964690cd-746f-40d9-867b-0913139c5e63 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:02.981305 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964690cd-746f-40d9-867b-0913139c5e63 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:02.984752 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964690cd-746f-40d9-867b-0913139c5e63 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:03.020541 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 469/947] 10.222.0.85 () {56 vars in 1149 bytes} [Tue Jun 7 14:21:02 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:21:03.030437 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964690cd-746f-40d9-867b-0913139c5e63 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:03.032297 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ed4fed-359f-4d29-80ee-eaca5f32a3c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:03.033119 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ed4fed-359f-4d29-80ee-eaca5f32a3c7 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:03.033553 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ed4fed-359f-4d29-80ee-eaca5f32a3c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:03.034039 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ed4fed-359f-4d29-80ee-eaca5f32a3c7 None admin] PATH_INFO: `/v3/services` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:03.035149 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ed4fed-359f-4d29-80ee-eaca5f32a3c7 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:03.035560 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ed4fed-359f-4d29-80ee-eaca5f32a3c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:03.038281 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ed4fed-359f-4d29-80ee-eaca5f32a3c7 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:03.050578 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 479/948] 10.222.0.85 () {56 vars in 1122 bytes} [Tue Jun 7 14:21:02 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:21:03.051772 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ed4fed-359f-4d29-80ee-eaca5f32a3c7 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:03.171750 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964690cd-746f-40d9-867b-0913139c5e63 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:03.184135 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ed4fed-359f-4d29-80ee-eaca5f32a3c7 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:03.219774 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546c7d8f-c7f6-41d0-89e0-5be36c06521c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:03.220797 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546c7d8f-c7f6-41d0-89e0-5be36c06521c None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:03.221087 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546c7d8f-c7f6-41d0-89e0-5be36c06521c None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:03.221367 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546c7d8f-c7f6-41d0-89e0-5be36c06521c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:03.223013 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd8ac26-708a-48c0-8d3a-fd202b426020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:03.224030 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd8ac26-708a-48c0-8d3a-fd202b426020 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:03.224461 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd8ac26-708a-48c0-8d3a-fd202b426020 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:03.225434 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd8ac26-708a-48c0-8d3a-fd202b426020 None admin] PATH_INFO: `/v3/services` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:03.226251 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd8ac26-708a-48c0-8d3a-fd202b426020 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:03.226660 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd8ac26-708a-48c0-8d3a-fd202b426020 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:03.228147 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd8ac26-708a-48c0-8d3a-fd202b426020 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:03.234675 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546c7d8f-c7f6-41d0-89e0-5be36c06521c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:03.236141 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546c7d8f-c7f6-41d0-89e0-5be36c06521c None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:03.238949 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 480/949] 10.222.0.85 () {56 vars in 1122 bytes} [Tue Jun 7 14:21:03 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:21:03.239859 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd8ac26-708a-48c0-8d3a-fd202b426020 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:03.251401 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:03.260313 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:03.262160 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-546c7d8f-c7f6-41d0-89e0-5be36c06521c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:21:03.262160 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:21:03.262160 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:21:03.262160 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:21:03.262160 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:21:03.262160 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:21:03.262160 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:21:03.262160 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:21:03.262160 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:21:03.262160 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:21:03.262160 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:21:03.262160 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:21:03.262160 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:21:03.262160 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:21:03.262160 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:21:03.264043 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:21:03.264043 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:21:03.264043 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:21:03.264043 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:21:03.264043 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:21:03.264043 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:21:03.264043 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:21:03.264043 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:21:03.264043 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:21:03.264043 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:21:03.264043 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:21:03.264043 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:21:03.264043 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:21:03.264043 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:21:03.264043 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:21:03.264043 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:21:03.264043 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:21:03.265806 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:21:03.265806 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:21:03.265806 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:21:03.265806 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:21:03.265806 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 470/950] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:21:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:21:03.272804 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546c7d8f-c7f6-41d0-89e0-5be36c06521c None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:03.365610 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd8ac26-708a-48c0-8d3a-fd202b426020 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:03.393664 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546c7d8f-c7f6-41d0-89e0-5be36c06521c None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:03.405110 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd871c6c-5705-488e-86d2-76096195fad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:03.405941 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd871c6c-5705-488e-86d2-76096195fad5 None admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:03.406224 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd871c6c-5705-488e-86d2-76096195fad5 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:03.406498 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd871c6c-5705-488e-86d2-76096195fad5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:03.407245 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd871c6c-5705-488e-86d2-76096195fad5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:03.408749 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd871c6c-5705-488e-86d2-76096195fad5 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:03.430126 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e55c65a-2a09-4b9e-aa67-e9dd7baa55ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:03.433620 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e55c65a-2a09-4b9e-aa67-e9dd7baa55ae None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:03.433620 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e55c65a-2a09-4b9e-aa67-e9dd7baa55ae None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:03.433620 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e55c65a-2a09-4b9e-aa67-e9dd7baa55ae None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:03.433620 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e55c65a-2a09-4b9e-aa67-e9dd7baa55ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:03.433620 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e55c65a-2a09-4b9e-aa67-e9dd7baa55ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:03.434083 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e55c65a-2a09-4b9e-aa67-e9dd7baa55ae None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:03.446582 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 471/951] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jun 7 14:21:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:21:03.451759 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e55c65a-2a09-4b9e-aa67-e9dd7baa55ae None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:03.514420 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:03.522600 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 481/952] 10.222.0.85 () {60 vars in 1144 bytes} [Tue Jun 7 14:21:03 2022] POST /identity/v3/endpoints => generated 349 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:21:03.576123 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e55c65a-2a09-4b9e-aa67-e9dd7baa55ae None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:03.621552 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b4887d-0b2f-43a6-8cd5-16e121874e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:03.622336 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b4887d-0b2f-43a6-8cd5-16e121874e0d None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:03.622563 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c9b6ca-cc2b-4ad4-9404-6fccf37b764b None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:03.623013 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b4887d-0b2f-43a6-8cd5-16e121874e0d None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:03.623090 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c9b6ca-cc2b-4ad4-9404-6fccf37b764b None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:03.623165 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b4887d-0b2f-43a6-8cd5-16e121874e0d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:03.623239 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c9b6ca-cc2b-4ad4-9404-6fccf37b764b None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:03.625938 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 482/953] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:21:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 14:21:03.630571 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef47b4a-c6b5-4612-9d1b-16efae8a7059 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:03.631424 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef47b4a-c6b5-4612-9d1b-16efae8a7059 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:03.631589 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b4887d-0b2f-43a6-8cd5-16e121874e0d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:03.631693 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef47b4a-c6b5-4612-9d1b-16efae8a7059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:03.633096 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b4887d-0b2f-43a6-8cd5-16e121874e0d None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:03.651004 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-3ef47b4a-c6b5-4612-9d1b-16efae8a7059 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:03.651772 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-c1b4887d-0b2f-43a6-8cd5-16e121874e0d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:21:03.651772 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:21:03.651772 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:21:03.651772 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:21:03.651772 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:21:03.651772 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:21:03.651772 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:21:03.651772 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:21:03.651772 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:21:03.651772 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:21:03.651772 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:21:03.651772 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:21:03.651772 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 14:21:03.651772 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 14:21:03.651772 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 14:21:03.652719 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 14:21:03.652719 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:21:03.652719 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:21:03.652719 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:21:03.652719 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:21:03.652719 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:21:03.652719 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:21:03.652719 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:21:03.652719 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:21:03.652719 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:21:03.652719 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:21:03.652719 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:21:03.652719 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:21:03.652719 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:21:03.652719 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:21:03.652719 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:21:03.652719 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:21:03.654245 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 14:21:03.654245 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 14:21:03.654245 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 14:21:03.654245 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 14:21:03.654245 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 14:21:03.654245 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 14:21:03.654245 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:21:03.654245 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:21:03.654245 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 472/954] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jun 7 14:21:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:21:03.654884 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-3ef47b4a-c6b5-4612-9d1b-16efae8a7059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:03.659288 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b4887d-0b2f-43a6-8cd5-16e121874e0d None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:03.733683 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 483/955] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:21:03 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 14:21:03.756233 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef47b4a-c6b5-4612-9d1b-16efae8a7059 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:03.758428 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:03.786368 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b4887d-0b2f-43a6-8cd5-16e121874e0d None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:03.808567 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:03.833685 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e0b3a1-0afa-4143-8fc4-6b3767ebfbe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:03.834946 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e0b3a1-0afa-4143-8fc4-6b3767ebfbe6 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:03.835235 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e0b3a1-0afa-4143-8fc4-6b3767ebfbe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:03.835516 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e0b3a1-0afa-4143-8fc4-6b3767ebfbe6 None admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:03.836583 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e0b3a1-0afa-4143-8fc4-6b3767ebfbe6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:03.836909 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e0b3a1-0afa-4143-8fc4-6b3767ebfbe6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:03.839155 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e0b3a1-0afa-4143-8fc4-6b3767ebfbe6 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:03.859402 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 473/956] 10.222.0.85 () {56 vars in 1148 bytes} [Tue Jun 7 14:21:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:21:03.866816 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e0b3a1-0afa-4143-8fc4-6b3767ebfbe6 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:03.887073 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef47b4a-c6b5-4612-9d1b-16efae8a7059 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:03.926505 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a54091d-b308-4613-ab7e-ac57fd31c90f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:03.927413 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a54091d-b308-4613-ab7e-ac57fd31c90f None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:03.928238 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a54091d-b308-4613-ab7e-ac57fd31c90f None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:03.928654 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a54091d-b308-4613-ab7e-ac57fd31c90f None admin] PATH_INFO: `/v3/services/compute` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:03.929327 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a54091d-b308-4613-ab7e-ac57fd31c90f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:03.930825 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a54091d-b308-4613-ab7e-ac57fd31c90f None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:03.952398 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-6a54091d-b308-4613-ab7e-ac57fd31c90f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 07 14:21:03.952398 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:21:03.952398 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 14:21:03.952398 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 14:21:03.952398 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 14:21:03.952398 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 14:21:03.952398 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 14:21:03.952398 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:21:03.952398 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 07 14:21:03.952398 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:21:03.952398 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 14:21:03.952398 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:21:03.952398 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:21:03.952398 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:21:03.952398 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:21:03.952398 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:21:03.952398 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:21:03.953552 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:21:03.953552 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:21:03.953552 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:21:03.953552 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:21:03.953552 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:21:03.953552 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:21:03.953552 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 14:21:03.953552 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 14:21:03.953552 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 14:21:03.953552 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 14:21:03.953552 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:21:03.953552 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:21:03.953552 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:21:03.953552 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:21:03.953552 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:21:03.953552 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:21:03.954411 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:21:03.954411 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:21:03.954411 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:21:03.954411 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:21:03.954411 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:21:03.954411 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:21:03.954411 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:21:03.954411 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:21:03.954411 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:21:03.954411 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:21:03.954411 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 14:21:03.954411 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:21:03.954411 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 07 14:21:03.954411 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:21:03.954411 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 484/957] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jun 7 14:21:03 2022] GET /identity/v3/services/compute => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:21:03.955221 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a54091d-b308-4613-ab7e-ac57fd31c90f None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:03.991202 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e0b3a1-0afa-4143-8fc4-6b3767ebfbe6 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:04.036601 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700ff2ca-5cac-4c67-bc25-8e42d3a9c123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:04.037508 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700ff2ca-5cac-4c67-bc25-8e42d3a9c123 None admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:04.037803 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700ff2ca-5cac-4c67-bc25-8e42d3a9c123 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:04.038384 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700ff2ca-5cac-4c67-bc25-8e42d3a9c123 None admin] PATH_INFO: `/v3/projects/724785eb74e44daf8736fd01fc0d1839/users/ad9d021b171244d796c320aa9f8450a9/roles/5ca16aa9be5740d2902e27a98242edf4` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:04.041294 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700ff2ca-5cac-4c67-bc25-8e42d3a9c123 None admin] RBAC: Authorizing `identity:create_grant(project_id=724785eb74e44daf8736fd01fc0d1839, user_id=ad9d021b171244d796c320aa9f8450a9, role_id=5ca16aa9be5740d2902e27a98242edf4)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:04.043121 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700ff2ca-5cac-4c67-bc25-8e42d3a9c123 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:04.044903 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:04.081941 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a54091d-b308-4613-ab7e-ac57fd31c90f None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:04.119528 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5b18d3-8d5c-47c4-b48e-362b8b554556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:04.119528 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5b18d3-8d5c-47c4-b48e-362b8b554556 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:04.119528 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5b18d3-8d5c-47c4-b48e-362b8b554556 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:04.120164 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5b18d3-8d5c-47c4-b48e-362b8b554556 None admin] PATH_INFO: `/v3/services` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:04.120752 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5b18d3-8d5c-47c4-b48e-362b8b554556 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:04.121062 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5b18d3-8d5c-47c4-b48e-362b8b554556 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:04.122490 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5b18d3-8d5c-47c4-b48e-362b8b554556 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:04.128599 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 474/958] 10.222.0.85 () {58 vars in 1439 bytes} [Tue Jun 7 14:21:03 2022] PUT /identity/v3/projects/724785eb74e44daf8736fd01fc0d1839/users/ad9d021b171244d796c320aa9f8450a9/roles/5ca16aa9be5740d2902e27a98242edf4 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:21:04.132925 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 485/959] 10.222.0.85 () {56 vars in 1112 bytes} [Tue Jun 7 14:21:03 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:21:04.134415 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700ff2ca-5cac-4c67-bc25-8e42d3a9c123 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:04.184418 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:04.258468 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700ff2ca-5cac-4c67-bc25-8e42d3a9c123 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:04.297632 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89b81cf-172b-4e39-a774-fb1b8fc1bfee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:04.298382 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b81cf-172b-4e39-a774-fb1b8fc1bfee None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:04.298702 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b81cf-172b-4e39-a774-fb1b8fc1bfee None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:04.298946 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b81cf-172b-4e39-a774-fb1b8fc1bfee None admin] PATH_INFO: `/v3/services` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:04.299672 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89b81cf-172b-4e39-a774-fb1b8fc1bfee None admin] RBAC: Adding query filter params (type=compute) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:04.300020 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89b81cf-172b-4e39-a774-fb1b8fc1bfee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:04.301527 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89b81cf-172b-4e39-a774-fb1b8fc1bfee None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:04.307464 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 475/960] 10.222.0.85 () {56 vars in 1112 bytes} [Tue Jun 7 14:21:04 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:21:05.040687 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce38c9a4-d957-4a46-87eb-2340d7017d38 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:05.041234 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce38c9a4-d957-4a46-87eb-2340d7017d38 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:05.041234 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce38c9a4-d957-4a46-87eb-2340d7017d38 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:05.042880 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 486/961] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:21:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:21:05.048091 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d351cb6-8af9-47d0-b106-b1e64ecca69f None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:05.048404 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d351cb6-8af9-47d0-b106-b1e64ecca69f None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:05.048689 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d351cb6-8af9-47d0-b106-b1e64ecca69f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:05.066284 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-9d351cb6-8af9-47d0-b106-b1e64ecca69f None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:05.067466 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-9d351cb6-8af9-47d0-b106-b1e64ecca69f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:05.138431 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 476/962] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:21:05 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 14:21:05.163185 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce38c9a4-d957-4a46-87eb-2340d7017d38 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:05.273977 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce38c9a4-d957-4a46-87eb-2340d7017d38 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:05.304102 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:05.310559 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239c8658-d076-4b88-9d90-456e4fd3de2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:05.311236 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239c8658-d076-4b88-9d90-456e4fd3de2d None admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:05.311520 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239c8658-d076-4b88-9d90-456e4fd3de2d None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:05.311797 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239c8658-d076-4b88-9d90-456e4fd3de2d None admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:05.312527 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239c8658-d076-4b88-9d90-456e4fd3de2d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:05.313954 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239c8658-d076-4b88-9d90-456e4fd3de2d None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:05.366848 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.sql.core [None req-239c8658-d076-4b88-9d90-456e4fd3de2d None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 07 14:21:05.366848 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 07 14:21:05.366848 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [parameters: {'user_id': 'd68d0f81a4d64369b3910a68f57d0d68', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 07 14:21:05.366848 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80387) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 07 14:21:05.375407 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-239c8658-d076-4b88-9d90-456e4fd3de2d None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 07 14:21:05.375407 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:21:05.375407 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 07 14:21:05.375407 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 07 14:21:05.375407 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 07 14:21:05.375407 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 07 14:21:05.375407 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 07 14:21:05.375407 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application result = self._query(query) Jun 07 14:21:05.375407 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 07 14:21:05.375407 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application conn.query(q) Jun 07 14:21:05.375407 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 07 14:21:05.375407 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 07 14:21:05.375407 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 07 14:21:05.375407 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application result.read() Jun 07 14:21:05.375407 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 07 14:21:05.376301 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 07 14:21:05.376301 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 07 14:21:05.376301 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 07 14:21:05.376301 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 07 14:21:05.376301 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 07 14:21:05.376301 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 07 14:21:05.376301 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 07 14:21:05.376301 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 07 14:21:05.376301 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:21:05.376301 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 07 14:21:05.376301 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:21:05.376301 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:21:05.376301 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 07 14:21:05.376301 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 07 14:21:05.376301 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 07 14:21:05.376301 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 07 14:21:05.376301 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 07 14:21:05.376301 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application next(self.gen) Jun 07 14:21:05.377299 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 07 14:21:05.377299 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application yield resource Jun 07 14:21:05.377299 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 07 14:21:05.377299 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application next(self.gen) Jun 07 14:21:05.377299 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 07 14:21:05.377299 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application self.session.rollback() Jun 07 14:21:05.377299 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 07 14:21:05.377299 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application self.force_reraise() Jun 07 14:21:05.377299 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 07 14:21:05.377299 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise self.value Jun 07 14:21:05.377299 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 07 14:21:05.377299 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 07 14:21:05.377299 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 07 14:21:05.377299 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application session.commit() Jun 07 14:21:05.377299 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 07 14:21:05.377299 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 07 14:21:05.377299 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 07 14:21:05.379762 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application self._prepare_impl() Jun 07 14:21:05.379762 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 07 14:21:05.379762 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application self.session.flush() Jun 07 14:21:05.379762 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 07 14:21:05.379762 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application self._flush(objects) Jun 07 14:21:05.379762 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 07 14:21:05.379762 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 07 14:21:05.379762 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 07 14:21:05.379762 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application compat.raise_( Jun 07 14:21:05.379762 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 07 14:21:05.379762 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception Jun 07 14:21:05.379762 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 07 14:21:05.379762 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application flush_context.execute() Jun 07 14:21:05.379762 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 07 14:21:05.379762 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rec.execute(self) Jun 07 14:21:05.379762 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 07 14:21:05.379762 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 07 14:21:05.391787 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 07 14:21:05.391787 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 07 14:21:05.391787 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 07 14:21:05.391787 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 07 14:21:05.391787 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 07 14:21:05.391787 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 07 14:21:05.391787 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 07 14:21:05.391787 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 07 14:21:05.391787 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 07 14:21:05.391787 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 07 14:21:05.391787 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 07 14:21:05.391787 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 07 14:21:05.391787 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 07 14:21:05.391787 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 07 14:21:05.391787 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 07 14:21:05.391787 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception Jun 07 14:21:05.392915 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 07 14:21:05.392915 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 07 14:21:05.392915 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 07 14:21:05.392915 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 07 14:21:05.392915 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 07 14:21:05.392915 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application result = self._query(query) Jun 07 14:21:05.392915 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 07 14:21:05.392915 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application conn.query(q) Jun 07 14:21:05.392915 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 07 14:21:05.392915 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 07 14:21:05.392915 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 07 14:21:05.392915 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application result.read() Jun 07 14:21:05.392915 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 07 14:21:05.392915 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 07 14:21:05.392915 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 07 14:21:05.392915 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 07 14:21:05.393956 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 07 14:21:05.393956 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 07 14:21:05.393956 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 07 14:21:05.393956 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 07 14:21:05.393956 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 07 14:21:05.393956 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 07 14:21:05.393956 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [parameters: {'user_id': 'd68d0f81a4d64369b3910a68f57d0d68', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 07 14:21:05.393956 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 07 14:21:05.393956 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:21:05.393956 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 14:21:05.393956 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:21:05.393956 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:21:05.393956 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:21:05.393956 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:21:05.393956 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:21:05.394976 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:21:05.394976 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:21:05.394976 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:21:05.394976 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:21:05.394976 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:21:05.394976 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:21:05.394976 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:21:05.394976 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 07 14:21:05.394976 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 07 14:21:05.394976 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:21:05.394976 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:21:05.394976 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 14:21:05.394976 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:21:05.394976 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 14:21:05.394976 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:21:05.394976 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 07 14:21:05.394976 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 07 14:21:05.395888 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 07 14:21:05.395888 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 07 14:21:05.395888 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 07 14:21:05.395888 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 07 14:21:05.395888 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 07 14:21:05.395888 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:21:05.395888 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 487/963] 10.222.0.85 () {60 vars in 1131 bytes} [Tue Jun 7 14:21:05 2022] POST /identity/v3/users => generated 160 bytes in 215 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:21:05.396332 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d351cb6-8af9-47d0-b106-b1e64ecca69f None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:05.420572 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:05.494730 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d351cb6-8af9-47d0-b106-b1e64ecca69f None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:05.530680 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7886f182-1e01-41fc-918f-c2d1414e9b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:05.531403 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7886f182-1e01-41fc-918f-c2d1414e9b9b None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:05.531687 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7886f182-1e01-41fc-918f-c2d1414e9b9b None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:05.531961 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7886f182-1e01-41fc-918f-c2d1414e9b9b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:05.539700 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7886f182-1e01-41fc-918f-c2d1414e9b9b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:05.541240 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7886f182-1e01-41fc-918f-c2d1414e9b9b None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:05.556367 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-7886f182-1e01-41fc-918f-c2d1414e9b9b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 07 14:21:05.556367 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:21:05.556367 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:21:05.556367 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:21:05.556367 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:21:05.556367 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:21:05.556367 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:21:05.556367 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:21:05.556367 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:21:05.556367 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:21:05.556367 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:21:05.556367 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:21:05.556367 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 14:21:05.556367 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 14:21:05.556367 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 14:21:05.556367 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 14:21:05.558666 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:21:05.558666 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:21:05.558666 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 14:21:05.558666 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:21:05.558666 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 14:21:05.558666 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:21:05.558666 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:21:05.558666 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:21:05.558666 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:21:05.558666 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:21:05.558666 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:21:05.558666 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:21:05.558666 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:21:05.558666 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:21:05.558666 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:21:05.558666 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:21:05.558666 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:21:05.558666 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:21:05.560994 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:21:05.560994 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:21:05.560994 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 14:21:05.560994 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 14:21:05.560994 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 14:21:05.560994 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 14:21:05.560994 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 14:21:05.560994 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 14:21:05.560994 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 07 14:21:05.560994 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:21:05.560994 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 477/964] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jun 7 14:21:05 2022] GET /identity/v3/users/cinder => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:21:05.564762 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239c8658-d076-4b88-9d90-456e4fd3de2d None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:05.678854 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239c8658-d076-4b88-9d90-456e4fd3de2d None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:05.715121 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4cb561-4f96-4b24-8d70-32d51750951a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:05.715800 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4cb561-4f96-4b24-8d70-32d51750951a None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:05.716089 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4cb561-4f96-4b24-8d70-32d51750951a None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:05.716396 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4cb561-4f96-4b24-8d70-32d51750951a None admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:05.717210 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4cb561-4f96-4b24-8d70-32d51750951a None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:05.717526 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4cb561-4f96-4b24-8d70-32d51750951a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:05.719194 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4cb561-4f96-4b24-8d70-32d51750951a None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:05.762782 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 488/965] 10.222.0.85 () {56 vars in 1101 bytes} [Tue Jun 7 14:21:05 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:21:05.993070 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac2250e-1dfb-4c53-9afe-0da33f0a5d5a None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:05.993388 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac2250e-1dfb-4c53-9afe-0da33f0a5d5a None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:05.993643 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac2250e-1dfb-4c53-9afe-0da33f0a5d5a None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:05.995559 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 478/966] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:21:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:21:06.001249 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023f1aed-cf50-4bfe-8ab1-3f5b4d940cce None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:06.001546 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023f1aed-cf50-4bfe-8ab1-3f5b4d940cce None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:06.001834 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023f1aed-cf50-4bfe-8ab1-3f5b4d940cce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:06.020521 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-023f1aed-cf50-4bfe-8ab1-3f5b4d940cce None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:06.022571 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-023f1aed-cf50-4bfe-8ab1-3f5b4d940cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:06.075157 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:06.098126 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 489/967] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:21:05 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:21:06.123011 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac2250e-1dfb-4c53-9afe-0da33f0a5d5a None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:06.216413 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:06.227398 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac2250e-1dfb-4c53-9afe-0da33f0a5d5a None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:06.269998 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee3cbd3-4672-46e5-95cc-187f489376ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:06.270867 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee3cbd3-4672-46e5-95cc-187f489376ab None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:06.271663 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee3cbd3-4672-46e5-95cc-187f489376ab None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:06.271949 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee3cbd3-4672-46e5-95cc-187f489376ab None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:06.272628 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee3cbd3-4672-46e5-95cc-187f489376ab None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:06.273998 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee3cbd3-4672-46e5-95cc-187f489376ab None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:06.286452 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-9ee3cbd3-4672-46e5-95cc-187f489376ab None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 14:21:06.286452 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:21:06.286452 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 14:21:06.286452 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 14:21:06.286452 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 14:21:06.286452 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 14:21:06.286452 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 14:21:06.286452 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:21:06.286452 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 14:21:06.286452 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:21:06.286452 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 14:21:06.286452 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:21:06.286452 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:21:06.286452 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:21:06.286452 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:21:06.286452 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:21:06.286452 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:21:06.288115 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:21:06.288115 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:21:06.288115 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:21:06.288115 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:21:06.288115 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:21:06.288115 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:21:06.288115 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 14:21:06.288115 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 14:21:06.288115 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 14:21:06.288115 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 14:21:06.288115 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:21:06.288115 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:21:06.288115 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:21:06.288115 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:21:06.288115 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:21:06.288115 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:21:06.293989 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:21:06.293989 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:21:06.293989 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:21:06.293989 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:21:06.293989 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:21:06.293989 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:21:06.293989 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:21:06.293989 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:21:06.293989 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:21:06.293989 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:21:06.293989 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 14:21:06.293989 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:21:06.293989 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 14:21:06.293989 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:21:06.293989 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 479/968] 10.222.0.85 () {56 vars in 1126 bytes} [Tue Jun 7 14:21:06 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:21:06.294806 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023f1aed-cf50-4bfe-8ab1-3f5b4d940cce None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:06.400155 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023f1aed-cf50-4bfe-8ab1-3f5b4d940cce None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:06.434413 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606e5449-7b65-4eca-b4eb-468da4fda1cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:06.435079 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606e5449-7b65-4eca-b4eb-468da4fda1cc None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:06.435363 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606e5449-7b65-4eca-b4eb-468da4fda1cc None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:06.435643 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606e5449-7b65-4eca-b4eb-468da4fda1cc None admin] PATH_INFO: `/v3/services` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:06.436351 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606e5449-7b65-4eca-b4eb-468da4fda1cc None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:06.436752 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606e5449-7b65-4eca-b4eb-468da4fda1cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:06.438129 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606e5449-7b65-4eca-b4eb-468da4fda1cc None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:06.447790 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 490/969] 10.222.0.85 () {56 vars in 1122 bytes} [Tue Jun 7 14:21:06 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:21:06.448763 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee3cbd3-4672-46e5-95cc-187f489376ab None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:06.556004 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee3cbd3-4672-46e5-95cc-187f489376ab None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:06.590053 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e1aabf-afed-44a5-a778-cccd95296493 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:06.590341 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e1aabf-afed-44a5-a778-cccd95296493 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:06.590622 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e1aabf-afed-44a5-a778-cccd95296493 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:06.592123 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6032ac6-e20a-4cd9-b800-d05cf8069488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:06.592301 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 491/970] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:21:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:21:06.592673 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6032ac6-e20a-4cd9-b800-d05cf8069488 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:06.592964 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6032ac6-e20a-4cd9-b800-d05cf8069488 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:06.593246 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6032ac6-e20a-4cd9-b800-d05cf8069488 None admin] PATH_INFO: `/v3/services` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:06.593969 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6032ac6-e20a-4cd9-b800-d05cf8069488 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:06.594283 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6032ac6-e20a-4cd9-b800-d05cf8069488 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:06.595778 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6032ac6-e20a-4cd9-b800-d05cf8069488 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:06.596931 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f508c8-cae1-46e6-ade2-ad0e9c915d4f None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:06.597222 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f508c8-cae1-46e6-ade2-ad0e9c915d4f None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:06.597503 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f508c8-cae1-46e6-ade2-ad0e9c915d4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:06.601531 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 480/971] 10.222.0.85 () {56 vars in 1122 bytes} [Tue Jun 7 14:21:06 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:21:06.607162 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6032ac6-e20a-4cd9-b800-d05cf8069488 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:06.613999 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-b8f508c8-cae1-46e6-ade2-ad0e9c915d4f None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:06.615200 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-b8f508c8-cae1-46e6-ade2-ad0e9c915d4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:06.629507 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:06.657533 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:06.683501 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 492/972] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:21:06 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 14:21:06.684896 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019de5f1-dfd5-4c19-b3b0-7afdb421b685 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:06.685185 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019de5f1-dfd5-4c19-b3b0-7afdb421b685 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:06.685465 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019de5f1-dfd5-4c19-b3b0-7afdb421b685 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:06.690815 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 493/973] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:21:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:21:06.692211 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b77ccf-6ce4-40e9-8062-de7e332073e6 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:06.692601 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b77ccf-6ce4-40e9-8062-de7e332073e6 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:06.692879 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b77ccf-6ce4-40e9-8062-de7e332073e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:06.711227 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-47b77ccf-6ce4-40e9-8062-de7e332073e6 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:06.712445 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-47b77ccf-6ce4-40e9-8062-de7e332073e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:06.717582 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6032ac6-e20a-4cd9-b800-d05cf8069488 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:06.752033 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616ba11c-ef14-4adb-bc6d-af1a4d6db82d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:06.752720 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ba11c-ef14-4adb-bc6d-af1a4d6db82d None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:06.753016 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ba11c-ef14-4adb-bc6d-af1a4d6db82d None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:06.753295 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ba11c-ef14-4adb-bc6d-af1a4d6db82d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:06.754099 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616ba11c-ef14-4adb-bc6d-af1a4d6db82d None admin] RBAC: Adding query filter params (interface=admin, service_id=b5b88f38fd6a4ac9b78bde30a831454e, region_id=RegionOne) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:06.754417 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616ba11c-ef14-4adb-bc6d-af1a4d6db82d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:06.756508 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616ba11c-ef14-4adb-bc6d-af1a4d6db82d None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:06.763126 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 481/974] 10.222.0.85 () {56 vars in 1249 bytes} [Tue Jun 7 14:21:06 2022] GET /identity/v3/endpoints?service_id=b5b88f38fd6a4ac9b78bde30a831454e&interface=admin®ion_id=RegionOne => generated 193 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:21:06.765246 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616ba11c-ef14-4adb-bc6d-af1a4d6db82d None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:06.780868 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 494/975] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:21:06 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:21:06.781706 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b77ccf-6ce4-40e9-8062-de7e332073e6 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:06.858433 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:06.877451 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616ba11c-ef14-4adb-bc6d-af1a4d6db82d None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:06.881441 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b77ccf-6ce4-40e9-8062-de7e332073e6 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:06.884545 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:06.910816 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695f73df-ae2a-40da-999d-7c76157d1401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:06.911728 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695f73df-ae2a-40da-999d-7c76157d1401 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:06.912043 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695f73df-ae2a-40da-999d-7c76157d1401 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:06.912318 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695f73df-ae2a-40da-999d-7c76157d1401 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:06.916682 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bfd4b5-2008-4459-9a76-9f8fad20201e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:06.917351 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bfd4b5-2008-4459-9a76-9f8fad20201e None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:06.917629 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bfd4b5-2008-4459-9a76-9f8fad20201e None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:06.917904 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bfd4b5-2008-4459-9a76-9f8fad20201e None admin] PATH_INFO: `/v3/services` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:06.918960 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bfd4b5-2008-4459-9a76-9f8fad20201e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:06.920206 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bfd4b5-2008-4459-9a76-9f8fad20201e None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:06.922885 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695f73df-ae2a-40da-999d-7c76157d1401 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:21:06.926142 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 495/976] 10.222.0.85 () {56 vars in 1087 bytes} [Tue Jun 7 14:21:06 2022] GET /identity/v3/services => generated 1497 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:21:06.926917 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bfd4b5-2008-4459-9a76-9f8fad20201e None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:06.931853 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-695f73df-ae2a-40da-999d-7c76157d1401 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 14:21:06.931853 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:21:06.931853 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:21:06.931853 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:21:06.931853 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:21:06.931853 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:21:06.931853 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:21:06.931853 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:21:06.931853 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:21:06.931853 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:21:06.931853 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:21:06.931853 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:21:06.931853 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:21:06.931853 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:21:06.931853 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:21:06.931853 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:21:06.932886 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:21:06.932886 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:21:06.932886 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:21:06.932886 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:21:06.932886 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:21:06.932886 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:21:06.932886 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:21:06.932886 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:21:06.932886 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:21:06.932886 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:21:06.932886 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:21:06.932886 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:21:06.932886 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:21:06.932886 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:21:06.932886 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:21:06.932886 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:21:06.933747 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:21:06.933747 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:21:06.933747 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:21:06.933747 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:21:06.933747 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:21:06.933747 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:21:06.933747 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:21:06.933747 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:21:06.933747 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:21:06.933747 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:21:06.933747 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 14:21:06.933747 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:21:06.938480 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 482/977] 10.222.0.85 () {56 vars in 1120 bytes} [Tue Jun 7 14:21:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:21:06.939459 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695f73df-ae2a-40da-999d-7c76157d1401 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:07.051677 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bfd4b5-2008-4459-9a76-9f8fad20201e None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:07.068556 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695f73df-ae2a-40da-999d-7c76157d1401 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:07.095291 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484134d8-1c20-4c6f-893f-ff61184494af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:07.096074 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484134d8-1c20-4c6f-893f-ff61184494af None admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:07.096365 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484134d8-1c20-4c6f-893f-ff61184494af None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:07.096663 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484134d8-1c20-4c6f-893f-ff61184494af None admin] PATH_INFO: `/v3/services` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:07.097408 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484134d8-1c20-4c6f-893f-ff61184494af None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:07.098809 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484134d8-1c20-4c6f-893f-ff61184494af None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:07.109990 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa418c1c-7a55-49e2-9aac-b868fee3b96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:07.110601 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa418c1c-7a55-49e2-9aac-b868fee3b96d None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:07.110890 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa418c1c-7a55-49e2-9aac-b868fee3b96d None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:07.111174 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa418c1c-7a55-49e2-9aac-b868fee3b96d None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:07.111939 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa418c1c-7a55-49e2-9aac-b868fee3b96d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:07.112278 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa418c1c-7a55-49e2-9aac-b868fee3b96d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:07.113694 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa418c1c-7a55-49e2-9aac-b868fee3b96d None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:07.123351 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 483/978] 10.222.0.85 () {56 vars in 1115 bytes} [Tue Jun 7 14:21:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:21:07.128870 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa418c1c-7a55-49e2-9aac-b868fee3b96d None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:07.132802 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:07.176182 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 496/979] 10.222.0.85 () {60 vars in 1141 bytes} [Tue Jun 7 14:21:06 2022] POST /identity/v3/services => generated 242 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:21:07.265647 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa418c1c-7a55-49e2-9aac-b868fee3b96d None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:07.319839 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cdeb72-bcff-4779-a0bf-e31bb77f598f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:07.320990 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cdeb72-bcff-4779-a0bf-e31bb77f598f None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:07.321499 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cdeb72-bcff-4779-a0bf-e31bb77f598f None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:07.321920 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cdeb72-bcff-4779-a0bf-e31bb77f598f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:07.341028 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cdeb72-bcff-4779-a0bf-e31bb77f598f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:07.342849 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cdeb72-bcff-4779-a0bf-e31bb77f598f None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:07.363649 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-b9cdeb72-bcff-4779-a0bf-e31bb77f598f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:21:07.363649 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:21:07.363649 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:21:07.363649 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:21:07.363649 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:21:07.363649 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:21:07.363649 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:21:07.363649 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:21:07.363649 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:21:07.363649 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:21:07.363649 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:21:07.363649 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:21:07.363649 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:21:07.363649 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:21:07.363649 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:21:07.364854 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:21:07.364854 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:21:07.364854 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:21:07.364854 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:21:07.364854 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:21:07.364854 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:21:07.364854 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:21:07.364854 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:21:07.364854 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:21:07.364854 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:21:07.364854 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:21:07.364854 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:21:07.364854 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:21:07.364854 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:21:07.364854 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:21:07.364854 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:21:07.364854 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:21:07.365785 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:21:07.365785 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:21:07.365785 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:21:07.365785 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:21:07.370909 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 484/980] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:21:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:21:07.372223 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484134d8-1c20-4c6f-893f-ff61184494af None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:07.407640 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:07.489738 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484134d8-1c20-4c6f-893f-ff61184494af None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:07.526517 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da89e0a6-c52f-4328-9a3b-8e7ab29f52ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:07.527226 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da89e0a6-c52f-4328-9a3b-8e7ab29f52ca None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:07.527512 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da89e0a6-c52f-4328-9a3b-8e7ab29f52ca None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:07.528014 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da89e0a6-c52f-4328-9a3b-8e7ab29f52ca None admin] PATH_INFO: `/v3/domains` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:07.528545 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da89e0a6-c52f-4328-9a3b-8e7ab29f52ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:07.528853 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da89e0a6-c52f-4328-9a3b-8e7ab29f52ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:07.530280 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da89e0a6-c52f-4328-9a3b-8e7ab29f52ca None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:07.542965 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 497/981] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jun 7 14:21:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:21:07.548683 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cdeb72-bcff-4779-a0bf-e31bb77f598f None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:07.608155 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:07.662555 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cdeb72-bcff-4779-a0bf-e31bb77f598f None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:07.700577 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cd345c-7316-4bb8-b69f-214253af3e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:07.701568 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cd345c-7316-4bb8-b69f-214253af3e00 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:07.701929 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cd345c-7316-4bb8-b69f-214253af3e00 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:07.702282 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cd345c-7316-4bb8-b69f-214253af3e00 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:07.710989 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cd345c-7316-4bb8-b69f-214253af3e00 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:07.712548 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cd345c-7316-4bb8-b69f-214253af3e00 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:07.727953 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-13cd345c-7316-4bb8-b69f-214253af3e00 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 14:21:07.727953 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:21:07.727953 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:21:07.727953 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:21:07.727953 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:21:07.727953 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:21:07.727953 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:21:07.727953 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:21:07.727953 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:21:07.727953 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:21:07.727953 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:21:07.727953 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:21:07.727953 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 14:21:07.727953 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 14:21:07.727953 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 14:21:07.727953 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 14:21:07.729181 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:21:07.729181 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:21:07.729181 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 14:21:07.729181 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:21:07.729181 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 14:21:07.729181 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:21:07.729181 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:21:07.729181 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:21:07.729181 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:21:07.729181 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:21:07.729181 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:21:07.729181 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:21:07.729181 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:21:07.729181 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:21:07.729181 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:21:07.729181 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:21:07.729181 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:21:07.729181 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:21:07.730343 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:21:07.730343 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:21:07.730343 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 14:21:07.730343 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 14:21:07.730343 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 14:21:07.730343 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 14:21:07.730343 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 14:21:07.730343 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 14:21:07.730343 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 14:21:07.730343 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:21:07.731652 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 485/982] 10.222.0.85 () {56 vars in 1117 bytes} [Tue Jun 7 14:21:07 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:21:07.737380 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da89e0a6-c52f-4328-9a3b-8e7ab29f52ca None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:07.834340 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:07.846663 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da89e0a6-c52f-4328-9a3b-8e7ab29f52ca None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:07.884925 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cada042-ddad-489d-b2db-b201467cb2f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:07.886165 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cada042-ddad-489d-b2db-b201467cb2f4 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:07.887400 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cada042-ddad-489d-b2db-b201467cb2f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:07.887400 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cada042-ddad-489d-b2db-b201467cb2f4 None admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:07.888664 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cada042-ddad-489d-b2db-b201467cb2f4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:07.889054 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cada042-ddad-489d-b2db-b201467cb2f4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:07.890963 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cada042-ddad-489d-b2db-b201467cb2f4 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:07.917228 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 498/983] 10.222.0.85 () {56 vars in 1149 bytes} [Tue Jun 7 14:21:07 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:21:07.922711 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cd345c-7316-4bb8-b69f-214253af3e00 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:08.043797 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cd345c-7316-4bb8-b69f-214253af3e00 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:08.058452 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:08.092991 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745b21f9-7052-4985-b96b-e8513e419223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:08.098728 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745b21f9-7052-4985-b96b-e8513e419223 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:08.099021 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745b21f9-7052-4985-b96b-e8513e419223 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:08.099301 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745b21f9-7052-4985-b96b-e8513e419223 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:08.179525 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745b21f9-7052-4985-b96b-e8513e419223 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:08.179525 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745b21f9-7052-4985-b96b-e8513e419223 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:08.289871 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 486/984] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jun 7 14:21:07 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 07 14:21:08.291150 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cada042-ddad-489d-b2db-b201467cb2f4 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:08.397178 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cada042-ddad-489d-b2db-b201467cb2f4 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:08.449522 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf6b5e5-f971-4577-9daf-6a2b3b7d3603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:08.450243 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf6b5e5-f971-4577-9daf-6a2b3b7d3603 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:08.450527 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf6b5e5-f971-4577-9daf-6a2b3b7d3603 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:08.450804 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf6b5e5-f971-4577-9daf-6a2b3b7d3603 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:08.461010 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf6b5e5-f971-4577-9daf-6a2b3b7d3603 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:08.462461 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf6b5e5-f971-4577-9daf-6a2b3b7d3603 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:08.469299 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:08.481039 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-caf6b5e5-f971-4577-9daf-6a2b3b7d3603 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:21:08.481039 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:21:08.481039 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:21:08.481039 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:21:08.481039 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:21:08.481039 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:21:08.481039 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:21:08.481039 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:21:08.481039 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:21:08.481039 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:21:08.481039 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:21:08.481039 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:21:08.481039 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:21:08.481039 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:21:08.481039 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:21:08.481963 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:21:08.481963 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:21:08.481963 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:21:08.481963 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:21:08.481963 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:21:08.481963 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:21:08.481963 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:21:08.481963 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:21:08.481963 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:21:08.481963 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:21:08.481963 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:21:08.481963 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:21:08.481963 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:21:08.481963 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:21:08.481963 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:21:08.481963 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:21:08.481963 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:21:08.482935 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:21:08.482935 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:21:08.482935 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:21:08.482935 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:21:08.483919 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 499/985] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:21:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:21:08.488156 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745b21f9-7052-4985-b96b-e8513e419223 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:08.509903 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:08.594766 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745b21f9-7052-4985-b96b-e8513e419223 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:08.629013 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88d6227-b961-4173-9ed6-bc6e1cf96df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:08.629677 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88d6227-b961-4173-9ed6-bc6e1cf96df2 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:08.629962 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88d6227-b961-4173-9ed6-bc6e1cf96df2 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:08.630240 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88d6227-b961-4173-9ed6-bc6e1cf96df2 None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:08.631522 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88d6227-b961-4173-9ed6-bc6e1cf96df2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:08.631830 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88d6227-b961-4173-9ed6-bc6e1cf96df2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:08.633247 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88d6227-b961-4173-9ed6-bc6e1cf96df2 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:08.649064 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 487/986] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jun 7 14:21:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:21:08.650490 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf6b5e5-f971-4577-9daf-6a2b3b7d3603 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:08.754199 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf6b5e5-f971-4577-9daf-6a2b3b7d3603 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:08.788945 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b70859-8fa9-422b-ab57-f402c61c4c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:08.789751 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b70859-8fa9-422b-ab57-f402c61c4c42 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:08.790036 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b70859-8fa9-422b-ab57-f402c61c4c42 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:08.790315 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b70859-8fa9-422b-ab57-f402c61c4c42 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:08.797168 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b70859-8fa9-422b-ab57-f402c61c4c42 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:08.798631 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b70859-8fa9-422b-ab57-f402c61c4c42 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:08.813044 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-e7b70859-8fa9-422b-ab57-f402c61c4c42 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:21:08.813044 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:21:08.813044 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:21:08.813044 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:21:08.813044 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:21:08.813044 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:21:08.813044 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:21:08.813044 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:21:08.813044 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:21:08.813044 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:21:08.813044 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:21:08.813044 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:21:08.813044 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 14:21:08.813044 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 14:21:08.813044 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 14:21:08.813966 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 14:21:08.813966 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:21:08.813966 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:21:08.813966 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:21:08.813966 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:21:08.813966 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:21:08.813966 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:21:08.813966 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:21:08.813966 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:21:08.813966 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:21:08.813966 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:21:08.813966 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:21:08.813966 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:21:08.813966 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:21:08.813966 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:21:08.813966 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:21:08.813966 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:21:08.814966 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 14:21:08.814966 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 14:21:08.814966 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 14:21:08.814966 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 14:21:08.814966 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 14:21:08.814966 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 14:21:08.814966 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:21:08.814966 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:21:08.816733 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 500/987] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jun 7 14:21:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:21:08.819941 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88d6227-b961-4173-9ed6-bc6e1cf96df2 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:08.895297 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:08.927252 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88d6227-b961-4173-9ed6-bc6e1cf96df2 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:08.969010 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e523f7a-f13b-4f6f-9a96-2c42848d14a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:08.969684 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e523f7a-f13b-4f6f-9a96-2c42848d14a8 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:08.969972 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e523f7a-f13b-4f6f-9a96-2c42848d14a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:08.970252 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e523f7a-f13b-4f6f-9a96-2c42848d14a8 None admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:08.971047 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e523f7a-f13b-4f6f-9a96-2c42848d14a8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:08.971361 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e523f7a-f13b-4f6f-9a96-2c42848d14a8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:08.972824 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e523f7a-f13b-4f6f-9a96-2c42848d14a8 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:08.991222 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 488/988] 10.222.0.85 () {56 vars in 1148 bytes} [Tue Jun 7 14:21:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:21:08.992630 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b70859-8fa9-422b-ab57-f402c61c4c42 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:09.053958 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:09.096883 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b70859-8fa9-422b-ab57-f402c61c4c42 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:09.132480 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03d6465-0814-4bee-9800-c727c9468fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:09.133155 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03d6465-0814-4bee-9800-c727c9468fbd None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:09.133441 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03d6465-0814-4bee-9800-c727c9468fbd None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:09.133837 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03d6465-0814-4bee-9800-c727c9468fbd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:09.134694 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03d6465-0814-4bee-9800-c727c9468fbd None admin] RBAC: Adding query filter params (role.id=5ca16aa9be5740d2902e27a98242edf4, scope.project.id=724785eb74e44daf8736fd01fc0d1839, user.id=ad9d021b171244d796c320aa9f8450a9) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:09.135008 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03d6465-0814-4bee-9800-c727c9468fbd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:09.136497 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03d6465-0814-4bee-9800-c727c9468fbd None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:09.143631 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 501/989] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jun 7 14:21:08 2022] GET /identity/v3/role_assignments?user.id=ad9d021b171244d796c320aa9f8450a9&scope.project.id=724785eb74e44daf8736fd01fc0d1839&role.id=5ca16aa9be5740d2902e27a98242edf4 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:21:09.220112 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd460af-fe6f-4dcf-913c-7b2545658273 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:09.220486 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd460af-fe6f-4dcf-913c-7b2545658273 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:09.220706 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd460af-fe6f-4dcf-913c-7b2545658273 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:09.222316 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 489/990] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:21:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:21:09.228486 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e83e38a-13e1-4798-8db3-7d530c5e9854 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:09.228842 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e83e38a-13e1-4798-8db3-7d530c5e9854 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:09.229220 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e83e38a-13e1-4798-8db3-7d530c5e9854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:09.247556 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-6e83e38a-13e1-4798-8db3-7d530c5e9854 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:09.248805 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-6e83e38a-13e1-4798-8db3-7d530c5e9854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:09.324204 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 502/991] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:21:09 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:21:09.347733 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd460af-fe6f-4dcf-913c-7b2545658273 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:09.456880 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd460af-fe6f-4dcf-913c-7b2545658273 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:09.485660 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da839f0-cce9-43f6-8093-2f9c743a3a8f None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:09.486007 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da839f0-cce9-43f6-8093-2f9c743a3a8f None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:09.486232 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da839f0-cce9-43f6-8093-2f9c743a3a8f None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:09.491312 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 503/992] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:21:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:21:09.492737 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16121d7d-63e1-4d87-b130-b1c2294988fb None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:09.493028 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16121d7d-63e1-4d87-b130-b1c2294988fb None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:09.493305 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16121d7d-63e1-4d87-b130-b1c2294988fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:09.494386 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:09.501548 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd3495d-5d49-4b2d-b14b-5df1a904466b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:09.502617 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd3495d-5d49-4b2d-b14b-5df1a904466b None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:09.502617 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd3495d-5d49-4b2d-b14b-5df1a904466b None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:09.502895 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd3495d-5d49-4b2d-b14b-5df1a904466b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:09.503550 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd3495d-5d49-4b2d-b14b-5df1a904466b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:09.506135 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd3495d-5d49-4b2d-b14b-5df1a904466b None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:09.510958 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-16121d7d-63e1-4d87-b130-b1c2294988fb None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:09.512133 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-16121d7d-63e1-4d87-b130-b1c2294988fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:09.519664 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-cdd3495d-5d49-4b2d-b14b-5df1a904466b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 14:21:09.519664 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:21:09.519664 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 14:21:09.519664 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 14:21:09.519664 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 14:21:09.519664 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 14:21:09.519664 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 14:21:09.519664 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:21:09.519664 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 14:21:09.519664 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:21:09.519664 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 14:21:09.519664 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:21:09.519664 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:21:09.519664 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:21:09.519664 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:21:09.519664 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:21:09.519664 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:21:09.520723 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:21:09.520723 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:21:09.520723 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:21:09.520723 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:21:09.520723 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:21:09.520723 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:21:09.520723 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 14:21:09.520723 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 14:21:09.520723 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 14:21:09.520723 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 14:21:09.520723 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:21:09.520723 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:21:09.520723 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:21:09.520723 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:21:09.520723 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:21:09.520723 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:21:09.525042 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:21:09.525042 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:21:09.525042 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:21:09.525042 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:21:09.525042 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:21:09.525042 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:21:09.525042 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:21:09.525042 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:21:09.525042 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:21:09.525042 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:21:09.525042 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 14:21:09.525042 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:21:09.525042 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 14:21:09.525042 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:21:09.525042 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 490/993] 10.222.0.85 () {56 vars in 1126 bytes} [Tue Jun 7 14:21:09 2022] GET /identity/v3/services/object-store => generated 93 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:21:09.527852 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd3495d-5d49-4b2d-b14b-5df1a904466b None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:09.567362 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:09.596543 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 504/994] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:21:09 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:21:09.617389 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16121d7d-63e1-4d87-b130-b1c2294988fb None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:09.641659 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd3495d-5d49-4b2d-b14b-5df1a904466b None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:09.677135 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8d522f-1693-4ff9-9be6-2629c30f87f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:09.677842 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d522f-1693-4ff9-9be6-2629c30f87f7 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:09.678131 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d522f-1693-4ff9-9be6-2629c30f87f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:09.678416 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d522f-1693-4ff9-9be6-2629c30f87f7 None admin] PATH_INFO: `/v3/services` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:09.679128 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8d522f-1693-4ff9-9be6-2629c30f87f7 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:09.679453 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8d522f-1693-4ff9-9be6-2629c30f87f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:09.680868 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8d522f-1693-4ff9-9be6-2629c30f87f7 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:09.685961 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 491/995] 10.222.0.85 () {56 vars in 1122 bytes} [Tue Jun 7 14:21:09 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:21:09.691981 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8d522f-1693-4ff9-9be6-2629c30f87f7 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:09.727566 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16121d7d-63e1-4d87-b130-b1c2294988fb None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:09.764519 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:09.770752 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f4c386-34a5-4dd2-b1e1-97ff4d261674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:09.772283 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f4c386-34a5-4dd2-b1e1-97ff4d261674 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:09.773108 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f4c386-34a5-4dd2-b1e1-97ff4d261674 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:09.774576 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f4c386-34a5-4dd2-b1e1-97ff4d261674 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:09.774576 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f4c386-34a5-4dd2-b1e1-97ff4d261674 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:09.776947 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f4c386-34a5-4dd2-b1e1-97ff4d261674 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:09.793209 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-e1f4c386-34a5-4dd2-b1e1-97ff4d261674 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 07 14:21:09.793209 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:21:09.793209 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 14:21:09.793209 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 14:21:09.793209 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 14:21:09.793209 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 14:21:09.793209 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 14:21:09.793209 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:21:09.793209 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 07 14:21:09.793209 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:21:09.793209 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 14:21:09.793209 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:21:09.793209 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:21:09.793209 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:21:09.793209 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:21:09.793209 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:21:09.793209 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:21:09.794535 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:21:09.794535 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:21:09.794535 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:21:09.794535 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:21:09.794535 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:21:09.794535 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:21:09.794535 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 14:21:09.794535 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 14:21:09.794535 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 14:21:09.794535 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 14:21:09.794535 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:21:09.794535 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:21:09.794535 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:21:09.794535 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:21:09.794535 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:21:09.794535 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:21:09.795681 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:21:09.795681 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:21:09.795681 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:21:09.795681 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:21:09.795681 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:21:09.795681 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:21:09.795681 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:21:09.795681 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:21:09.795681 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:21:09.795681 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:21:09.795681 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 14:21:09.795681 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:21:09.795681 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 07 14:21:09.795681 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:21:09.795681 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 505/996] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jun 7 14:21:09 2022] GET /identity/v3/services/compute => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:21:09.800687 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f4c386-34a5-4dd2-b1e1-97ff4d261674 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:09.814919 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8d522f-1693-4ff9-9be6-2629c30f87f7 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:09.850915 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:09.854469 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269e7450-6b35-413a-a17d-5957a076e610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:09.855147 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269e7450-6b35-413a-a17d-5957a076e610 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:09.855431 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269e7450-6b35-413a-a17d-5957a076e610 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:09.855710 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269e7450-6b35-413a-a17d-5957a076e610 None admin] PATH_INFO: `/v3/services` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:09.856458 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269e7450-6b35-413a-a17d-5957a076e610 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:09.856766 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269e7450-6b35-413a-a17d-5957a076e610 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:09.858183 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269e7450-6b35-413a-a17d-5957a076e610 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:09.864420 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 492/997] 10.222.0.85 () {56 vars in 1122 bytes} [Tue Jun 7 14:21:09 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:21:09.870830 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269e7450-6b35-413a-a17d-5957a076e610 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:09.912969 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f4c386-34a5-4dd2-b1e1-97ff4d261674 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:09.949351 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa74d11-1953-49b1-9ba9-b34b1730739a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:09.950762 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa74d11-1953-49b1-9ba9-b34b1730739a None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:09.950762 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa74d11-1953-49b1-9ba9-b34b1730739a None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:09.951051 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa74d11-1953-49b1-9ba9-b34b1730739a None admin] PATH_INFO: `/v3/services` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:09.951819 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa74d11-1953-49b1-9ba9-b34b1730739a None admin] RBAC: Adding query filter params (name=compute) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:09.953001 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa74d11-1953-49b1-9ba9-b34b1730739a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:09.954428 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa74d11-1953-49b1-9ba9-b34b1730739a None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:09.967665 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 506/998] 10.222.0.85 () {56 vars in 1112 bytes} [Tue Jun 7 14:21:09 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:21:09.968956 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa74d11-1953-49b1-9ba9-b34b1730739a None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:09.994012 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269e7450-6b35-413a-a17d-5957a076e610 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:10.020756 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:10.032730 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4143199-9d4d-47b7-a258-c2ade7a5ab8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:10.033398 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4143199-9d4d-47b7-a258-c2ade7a5ab8c None admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:10.033684 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4143199-9d4d-47b7-a258-c2ade7a5ab8c None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:10.033989 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4143199-9d4d-47b7-a258-c2ade7a5ab8c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:10.034818 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4143199-9d4d-47b7-a258-c2ade7a5ab8c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:10.036188 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4143199-9d4d-47b7-a258-c2ade7a5ab8c None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:10.081257 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa74d11-1953-49b1-9ba9-b34b1730739a None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:10.092346 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:10.118084 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818cf3bc-18c0-4378-b8c0-e35e1b41541e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:10.118679 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818cf3bc-18c0-4378-b8c0-e35e1b41541e None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:10.118967 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818cf3bc-18c0-4378-b8c0-e35e1b41541e None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:10.119247 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818cf3bc-18c0-4378-b8c0-e35e1b41541e None admin] PATH_INFO: `/v3/services` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:10.120214 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818cf3bc-18c0-4378-b8c0-e35e1b41541e None admin] RBAC: Adding query filter params (type=compute) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:10.120569 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818cf3bc-18c0-4378-b8c0-e35e1b41541e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:10.121938 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818cf3bc-18c0-4378-b8c0-e35e1b41541e None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:10.126726 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 507/999] 10.222.0.85 () {56 vars in 1112 bytes} [Tue Jun 7 14:21:09 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:21:10.133259 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818cf3bc-18c0-4378-b8c0-e35e1b41541e None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:10.156013 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 493/1000] 10.222.0.85 () {60 vars in 1144 bytes} [Tue Jun 7 14:21:09 2022] POST /identity/v3/endpoints => generated 325 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:21:10.250089 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818cf3bc-18c0-4378-b8c0-e35e1b41541e None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:10.299708 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555b5b4b-b13e-499d-92ac-8e2436efade9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:10.300396 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555b5b4b-b13e-499d-92ac-8e2436efade9 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:10.300723 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555b5b4b-b13e-499d-92ac-8e2436efade9 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:10.300961 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555b5b4b-b13e-499d-92ac-8e2436efade9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:10.301764 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555b5b4b-b13e-499d-92ac-8e2436efade9 None admin] RBAC: Adding query filter params (interface=public, service_id=5dc4e652d5424cc0ab9d7d5ed0f9c548, region_id=RegionOne) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:10.302076 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555b5b4b-b13e-499d-92ac-8e2436efade9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:10.304039 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555b5b4b-b13e-499d-92ac-8e2436efade9 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:10.314203 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 508/1001] 10.222.0.85 () {56 vars in 1251 bytes} [Tue Jun 7 14:21:10 2022] GET /identity/v3/endpoints?service_id=5dc4e652d5424cc0ab9d7d5ed0f9c548&interface=public®ion_id=RegionOne => generated 194 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:21:10.316082 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4143199-9d4d-47b7-a258-c2ade7a5ab8c None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:10.424918 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4143199-9d4d-47b7-a258-c2ade7a5ab8c None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:10.458753 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7198b9-df66-45c9-b7cd-21d6532e8b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:10.459420 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7198b9-df66-45c9-b7cd-21d6532e8b7b None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:10.459704 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7198b9-df66-45c9-b7cd-21d6532e8b7b None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:10.459983 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7198b9-df66-45c9-b7cd-21d6532e8b7b None admin] PATH_INFO: `/v3/services` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:10.460840 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7198b9-df66-45c9-b7cd-21d6532e8b7b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:10.462216 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7198b9-df66-45c9-b7cd-21d6532e8b7b None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:10.468808 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 494/1002] 10.222.0.85 () {56 vars in 1087 bytes} [Tue Jun 7 14:21:10 2022] GET /identity/v3/services => generated 1727 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:21:11.456890 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b44d29-38bc-47e4-b76b-87883996b4ab None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:11.457409 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b44d29-38bc-47e4-b76b-87883996b4ab None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:11.457487 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b44d29-38bc-47e4-b76b-87883996b4ab None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:11.459647 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 509/1003] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:21:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:21:11.465150 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad23b41b-4ad9-444d-aa7e-4ba25a8bdc4c None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:11.465656 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad23b41b-4ad9-444d-aa7e-4ba25a8bdc4c None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:11.465738 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad23b41b-4ad9-444d-aa7e-4ba25a8bdc4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:11.486382 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-ad23b41b-4ad9-444d-aa7e-4ba25a8bdc4c None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:11.488309 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-ad23b41b-4ad9-444d-aa7e-4ba25a8bdc4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:11.550441 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:11.575092 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 495/1004] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:21:11 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:21:11.598196 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b44d29-38bc-47e4-b76b-87883996b4ab None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:11.636364 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:11.715300 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b44d29-38bc-47e4-b76b-87883996b4ab None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:11.750592 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0383f944-d25e-48bf-bbcd-693a7dc9f0b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:11.751392 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0383f944-d25e-48bf-bbcd-693a7dc9f0b2 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:11.751678 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0383f944-d25e-48bf-bbcd-693a7dc9f0b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:11.751955 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0383f944-d25e-48bf-bbcd-693a7dc9f0b2 None admin] PATH_INFO: `/v3/services/image` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:11.752650 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0383f944-d25e-48bf-bbcd-693a7dc9f0b2 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:11.754010 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0383f944-d25e-48bf-bbcd-693a7dc9f0b2 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:11.771323 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-0383f944-d25e-48bf-bbcd-693a7dc9f0b2 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 07 14:21:11.771323 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:21:11.771323 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 14:21:11.771323 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 14:21:11.771323 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 14:21:11.771323 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 14:21:11.771323 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 14:21:11.771323 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:21:11.771323 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 07 14:21:11.771323 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:21:11.771323 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 14:21:11.771323 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:21:11.771323 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:21:11.771323 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:21:11.771323 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:21:11.771323 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:21:11.771323 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:21:11.772413 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:21:11.772413 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:21:11.772413 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:21:11.772413 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:21:11.772413 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:21:11.772413 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:21:11.772413 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 14:21:11.772413 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 14:21:11.772413 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 14:21:11.772413 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 14:21:11.772413 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:21:11.772413 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:21:11.772413 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:21:11.772413 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:21:11.772413 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:21:11.772413 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:21:11.773284 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:21:11.773284 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:21:11.773284 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:21:11.773284 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:21:11.773284 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:21:11.773284 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:21:11.773284 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:21:11.773284 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:21:11.773284 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:21:11.773284 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:21:11.773284 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 14:21:11.773284 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:21:11.773284 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 07 14:21:11.773284 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:21:11.773284 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 510/1005] 10.222.0.85 () {56 vars in 1105 bytes} [Tue Jun 7 14:21:11 2022] GET /identity/v3/services/image => generated 86 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:21:11.778679 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad23b41b-4ad9-444d-aa7e-4ba25a8bdc4c None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:11.881684 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad23b41b-4ad9-444d-aa7e-4ba25a8bdc4c None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:11.915596 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fc88f6-facb-478b-af76-55a38ba6a0fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:11.916270 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fc88f6-facb-478b-af76-55a38ba6a0fb None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:11.916583 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fc88f6-facb-478b-af76-55a38ba6a0fb None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:11.916866 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fc88f6-facb-478b-af76-55a38ba6a0fb None admin] PATH_INFO: `/v3/services` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:11.917575 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fc88f6-facb-478b-af76-55a38ba6a0fb None admin] RBAC: Adding query filter params (name=image) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:11.917883 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fc88f6-facb-478b-af76-55a38ba6a0fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:11.919241 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fc88f6-facb-478b-af76-55a38ba6a0fb None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:11.928446 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 496/1006] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:21:11 2022] GET /identity/v3/services?name=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:21:11.930446 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0383f944-d25e-48bf-bbcd-693a7dc9f0b2 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:12.033252 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:12.037696 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0383f944-d25e-48bf-bbcd-693a7dc9f0b2 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:12.076717 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8d0a06-f98a-4810-9430-6a1993b45af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:12.077394 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8d0a06-f98a-4810-9430-6a1993b45af8 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:12.077681 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8d0a06-f98a-4810-9430-6a1993b45af8 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:12.077949 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8d0a06-f98a-4810-9430-6a1993b45af8 None admin] PATH_INFO: `/v3/services` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:12.078665 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8d0a06-f98a-4810-9430-6a1993b45af8 None admin] RBAC: Adding query filter params (type=image) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:12.078986 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8d0a06-f98a-4810-9430-6a1993b45af8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:12.080345 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8d0a06-f98a-4810-9430-6a1993b45af8 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:12.085951 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 511/1007] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:21:11 2022] GET /identity/v3/services?type=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:21:12.475049 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe79ca5-3dfc-4974-9841-edc73b3623f4 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:12.475602 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe79ca5-3dfc-4974-9841-edc73b3623f4 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:12.475602 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe79ca5-3dfc-4974-9841-edc73b3623f4 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:12.477069 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 497/1008] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:21:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:21:12.482703 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6ce72f-c496-424c-9cfb-b18accef5f33 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:12.483427 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6ce72f-c496-424c-9cfb-b18accef5f33 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:12.483427 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6ce72f-c496-424c-9cfb-b18accef5f33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:12.501170 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-4f6ce72f-c496-424c-9cfb-b18accef5f33 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:12.502369 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-4f6ce72f-c496-424c-9cfb-b18accef5f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:12.578295 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 512/1009] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:21:12 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:21:12.604601 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe79ca5-3dfc-4974-9841-edc73b3623f4 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:12.672108 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:12.758197 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe79ca5-3dfc-4974-9841-edc73b3623f4 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:12.770896 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ced609c-5476-422b-b75f-debfe76d94c8 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:12.771893 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ced609c-5476-422b-b75f-debfe76d94c8 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:12.772181 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ced609c-5476-422b-b75f-debfe76d94c8 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:12.773881 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 513/1010] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:21:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 14:21:12.779755 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e001d306-8909-4a2d-9586-70a86627ddee None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:12.780143 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e001d306-8909-4a2d-9586-70a86627ddee None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:12.780719 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e001d306-8909-4a2d-9586-70a86627ddee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:12.800481 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacbab84-a3e6-4895-82b8-5c71d51f09e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:12.801302 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacbab84-a3e6-4895-82b8-5c71d51f09e8 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:12.801590 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacbab84-a3e6-4895-82b8-5c71d51f09e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:12.801899 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacbab84-a3e6-4895-82b8-5c71d51f09e8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:12.802910 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-e001d306-8909-4a2d-9586-70a86627ddee None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:12.803134 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacbab84-a3e6-4895-82b8-5c71d51f09e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:12.804130 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-e001d306-8909-4a2d-9586-70a86627ddee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:12.804628 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacbab84-a3e6-4895-82b8-5c71d51f09e8 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:12.806336 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 498/1011] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:21:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:21:12.812010 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacbab84-a3e6-4895-82b8-5c71d51f09e8 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:12.906768 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 514/1012] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:21:12 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:21:12.929408 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e001d306-8909-4a2d-9586-70a86627ddee None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:12.960189 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:12.961284 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacbab84-a3e6-4895-82b8-5c71d51f09e8 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:12.979478 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:13.014047 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bf4b12-0ef7-422c-a5c6-0accbe711024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:13.015528 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bf4b12-0ef7-422c-a5c6-0accbe711024 None admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:13.015996 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bf4b12-0ef7-422c-a5c6-0accbe711024 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:13.016939 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bf4b12-0ef7-422c-a5c6-0accbe711024 None admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:13.018695 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bf4b12-0ef7-422c-a5c6-0accbe711024 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:13.022542 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bf4b12-0ef7-422c-a5c6-0accbe711024 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:13.041267 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e001d306-8909-4a2d-9586-70a86627ddee None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:13.079295 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568aeef9-6812-464a-8838-b430f566e2b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:13.080105 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568aeef9-6812-464a-8838-b430f566e2b7 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:13.080419 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568aeef9-6812-464a-8838-b430f566e2b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:13.080712 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568aeef9-6812-464a-8838-b430f566e2b7 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:13.081369 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568aeef9-6812-464a-8838-b430f566e2b7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:13.082757 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568aeef9-6812-464a-8838-b430f566e2b7 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:13.096345 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-568aeef9-6812-464a-8838-b430f566e2b7 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 07 14:21:13.096345 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:21:13.096345 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 14:21:13.096345 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 14:21:13.096345 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 14:21:13.096345 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 14:21:13.096345 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 14:21:13.096345 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:21:13.096345 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 07 14:21:13.096345 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:21:13.096345 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 14:21:13.096345 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:21:13.096345 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:21:13.096345 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:21:13.096345 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:21:13.096345 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:21:13.096345 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:21:13.102641 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:21:13.102641 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:21:13.102641 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:21:13.102641 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:21:13.102641 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:21:13.102641 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:21:13.102641 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 14:21:13.102641 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 14:21:13.102641 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 14:21:13.102641 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 14:21:13.102641 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:21:13.102641 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:21:13.102641 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:21:13.102641 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:21:13.102641 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:21:13.102641 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:21:13.103526 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:21:13.103526 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:21:13.103526 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:21:13.103526 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:21:13.103526 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:21:13.103526 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:21:13.103526 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:21:13.103526 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:21:13.103526 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:21:13.103526 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:21:13.103526 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 14:21:13.103526 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:21:13.103526 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 07 14:21:13.103526 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:21:13.103526 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 515/1013] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jun 7 14:21:12 2022] GET /identity/v3/services/compute => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:21:13.104404 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568aeef9-6812-464a-8838-b430f566e2b7 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:13.178324 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 499/1014] 10.222.0.85 () {60 vars in 1141 bytes} [Tue Jun 7 14:21:12 2022] POST /identity/v3/projects => generated 311 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:21:13.204493 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:13.225565 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568aeef9-6812-464a-8838-b430f566e2b7 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:13.272731 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31f8769-67ba-49a2-92da-535c8dcb26c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:13.273349 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31f8769-67ba-49a2-92da-535c8dcb26c5 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:13.273641 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31f8769-67ba-49a2-92da-535c8dcb26c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:13.273929 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31f8769-67ba-49a2-92da-535c8dcb26c5 None admin] PATH_INFO: `/v3/services` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:13.274691 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31f8769-67ba-49a2-92da-535c8dcb26c5 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:13.275005 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31f8769-67ba-49a2-92da-535c8dcb26c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:13.276458 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31f8769-67ba-49a2-92da-535c8dcb26c5 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:13.287595 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 516/1015] 10.222.0.85 () {56 vars in 1112 bytes} [Tue Jun 7 14:21:13 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:21:13.288807 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bf4b12-0ef7-422c-a5c6-0accbe711024 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:13.411299 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bf4b12-0ef7-422c-a5c6-0accbe711024 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:13.451221 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f56d0f1-90a2-43c6-9146-e80846608c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:13.451916 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f56d0f1-90a2-43c6-9146-e80846608c6f None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:13.452200 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f56d0f1-90a2-43c6-9146-e80846608c6f None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:13.452523 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f56d0f1-90a2-43c6-9146-e80846608c6f None admin] PATH_INFO: `/v3/services` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:13.453261 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f56d0f1-90a2-43c6-9146-e80846608c6f None admin] RBAC: Adding query filter params (type=compute) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:13.453582 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f56d0f1-90a2-43c6-9146-e80846608c6f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:13.454993 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f56d0f1-90a2-43c6-9146-e80846608c6f None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:13.465624 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 500/1016] 10.222.0.85 () {56 vars in 1112 bytes} [Tue Jun 7 14:21:13 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:21:13.470462 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31f8769-67ba-49a2-92da-535c8dcb26c5 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:13.582286 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31f8769-67ba-49a2-92da-535c8dcb26c5 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:13.618831 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6aee71-1320-4b01-94ec-9ad56f78f746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:13.619503 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6aee71-1320-4b01-94ec-9ad56f78f746 None admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:13.619784 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6aee71-1320-4b01-94ec-9ad56f78f746 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:13.620322 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6aee71-1320-4b01-94ec-9ad56f78f746 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:13.621117 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6aee71-1320-4b01-94ec-9ad56f78f746 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:13.622481 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6aee71-1320-4b01-94ec-9ad56f78f746 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:13.716433 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 517/1017] 10.222.0.85 () {60 vars in 1144 bytes} [Tue Jun 7 14:21:13 2022] POST /identity/v3/endpoints => generated 334 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:21:14.621958 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b07a10-7490-4115-8a86-96ecb5709722 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:14.622524 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b07a10-7490-4115-8a86-96ecb5709722 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:14.622524 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b07a10-7490-4115-8a86-96ecb5709722 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:14.624071 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 501/1018] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:21:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:21:14.629606 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f369d16-265f-4332-b64b-8122f02026ca None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:14.630226 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f369d16-265f-4332-b64b-8122f02026ca None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:14.630226 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f369d16-265f-4332-b64b-8122f02026ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:14.643016 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:14.650494 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-4f369d16-265f-4332-b64b-8122f02026ca None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:14.651733 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-4f369d16-265f-4332-b64b-8122f02026ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:14.758815 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 518/1019] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:21:14 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:21:14.781526 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b07a10-7490-4115-8a86-96ecb5709722 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:14.813079 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:14.908112 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b07a10-7490-4115-8a86-96ecb5709722 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:14.945707 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd0992e-76b1-4dde-89d2-364c452877c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:14.946389 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd0992e-76b1-4dde-89d2-364c452877c7 None admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:14.946669 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd0992e-76b1-4dde-89d2-364c452877c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:14.946959 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd0992e-76b1-4dde-89d2-364c452877c7 None admin] PATH_INFO: `/v3/services` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:14.947718 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd0992e-76b1-4dde-89d2-364c452877c7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:14.950012 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd0992e-76b1-4dde-89d2-364c452877c7 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:15.026195 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 502/1020] 10.222.0.85 () {60 vars in 1141 bytes} [Tue Jun 7 14:21:14 2022] POST /identity/v3/services => generated 242 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:21:15.744577 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96febb97-2e6e-43b3-834e-a254cdb5ca9b None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:15.745407 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96febb97-2e6e-43b3-834e-a254cdb5ca9b None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:15.745407 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96febb97-2e6e-43b3-834e-a254cdb5ca9b None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:15.746779 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 519/1021] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:21:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:21:15.752808 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4156c7-f7ce-47f1-9780-6849f1e96d79 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:15.753297 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4156c7-f7ce-47f1-9780-6849f1e96d79 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:15.753402 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4156c7-f7ce-47f1-9780-6849f1e96d79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:15.770810 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-bb4156c7-f7ce-47f1-9780-6849f1e96d79 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:15.772561 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-bb4156c7-f7ce-47f1-9780-6849f1e96d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:15.868316 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 503/1022] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:21:15 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:21:15.894666 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96febb97-2e6e-43b3-834e-a254cdb5ca9b None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:16.009505 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96febb97-2e6e-43b3-834e-a254cdb5ca9b None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:16.043100 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:16.052360 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170a7537-876d-435e-8384-61e0187dc35e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:16.053207 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170a7537-876d-435e-8384-61e0187dc35e None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:16.053493 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170a7537-876d-435e-8384-61e0187dc35e None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:16.062525 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170a7537-876d-435e-8384-61e0187dc35e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:16.062525 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170a7537-876d-435e-8384-61e0187dc35e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:16.062525 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170a7537-876d-435e-8384-61e0187dc35e None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:16.062525 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 520/1023] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:21:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:21:16.064365 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4156c7-f7ce-47f1-9780-6849f1e96d79 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:16.081341 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:16.186490 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4156c7-f7ce-47f1-9780-6849f1e96d79 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:16.230741 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aae330-99e4-4b9d-ae5b-f94f41d3078d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:16.231553 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aae330-99e4-4b9d-ae5b-f94f41d3078d None admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:16.231843 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aae330-99e4-4b9d-ae5b-f94f41d3078d None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:16.232122 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aae330-99e4-4b9d-ae5b-f94f41d3078d None admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:16.232885 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5aae330-99e4-4b9d-ae5b-f94f41d3078d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:16.234324 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5aae330-99e4-4b9d-ae5b-f94f41d3078d None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:16.352647 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 504/1024] 10.222.0.85 () {60 vars in 1132 bytes} [Tue Jun 7 14:21:16 2022] POST /identity/v3/users => generated 285 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:21:16.446929 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c6cf1-d2cd-4a79-9916-b0173b051f11 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:16.447291 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c6cf1-d2cd-4a79-9916-b0173b051f11 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:16.447490 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c6cf1-d2cd-4a79-9916-b0173b051f11 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:16.449172 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 521/1025] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:21:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 14:21:16.454856 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bfa59e-bddd-4d33-9dfc-8584cd8e9a76 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:16.455161 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bfa59e-bddd-4d33-9dfc-8584cd8e9a76 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:16.455433 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bfa59e-bddd-4d33-9dfc-8584cd8e9a76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:16.477148 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-c3bfa59e-bddd-4d33-9dfc-8584cd8e9a76 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:16.478381 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-c3bfa59e-bddd-4d33-9dfc-8584cd8e9a76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:16.581506 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:16.589654 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 505/1026] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:21:16 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:21:16.615657 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5c6cf1-d2cd-4a79-9916-b0173b051f11 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:16.738610 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5c6cf1-d2cd-4a79-9916-b0173b051f11 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:16.778328 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9cd54b-fe90-4efc-be2e-d0745fd4f3d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:16.779165 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9cd54b-fe90-4efc-be2e-d0745fd4f3d6 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:16.779449 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9cd54b-fe90-4efc-be2e-d0745fd4f3d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:16.779727 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9cd54b-fe90-4efc-be2e-d0745fd4f3d6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:16.792780 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9cd54b-fe90-4efc-be2e-d0745fd4f3d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:21:16.802050 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-ec9cd54b-fe90-4efc-be2e-d0745fd4f3d6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 14:21:16.802050 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:21:16.802050 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:21:16.802050 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:21:16.802050 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:21:16.802050 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:21:16.802050 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:21:16.802050 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:21:16.802050 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:21:16.802050 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:21:16.802050 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:21:16.802050 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:21:16.802050 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:21:16.802050 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:21:16.802050 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:21:16.802050 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:21:16.803660 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:21:16.803660 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:21:16.803660 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:21:16.803660 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:21:16.803660 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:21:16.803660 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:21:16.803660 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:21:16.803660 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:21:16.803660 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:21:16.803660 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:21:16.803660 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:21:16.803660 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:21:16.803660 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:21:16.803660 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:21:16.803660 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:21:16.803660 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:21:16.811160 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:21:16.811160 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:21:16.811160 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:21:16.811160 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:21:16.811160 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:21:16.811160 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:21:16.811160 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:21:16.811160 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:21:16.811160 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:21:16.811160 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:21:16.811160 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 14:21:16.811160 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:21:16.811160 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 522/1027] 10.222.0.85 () {56 vars in 1120 bytes} [Tue Jun 7 14:21:16 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:21:16.811880 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bfa59e-bddd-4d33-9dfc-8584cd8e9a76 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:16.956433 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bfa59e-bddd-4d33-9dfc-8584cd8e9a76 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:16.998898 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6b06d4-a367-4a30-bbe6-d9b7588eaba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:16.999706 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b06d4-a367-4a30-bbe6-d9b7588eaba5 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:17.000249 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b06d4-a367-4a30-bbe6-d9b7588eaba5 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:17.001048 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b06d4-a367-4a30-bbe6-d9b7588eaba5 None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:17.002047 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6b06d4-a367-4a30-bbe6-d9b7588eaba5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:17.002366 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6b06d4-a367-4a30-bbe6-d9b7588eaba5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:17.003812 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6b06d4-a367-4a30-bbe6-d9b7588eaba5 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:17.019760 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 506/1028] 10.222.0.85 () {56 vars in 1115 bytes} [Tue Jun 7 14:21:16 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:21:17.025878 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9cd54b-fe90-4efc-be2e-d0745fd4f3d6 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:17.101819 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:17.154467 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9cd54b-fe90-4efc-be2e-d0745fd4f3d6 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:17.207433 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4cc6b0-22c3-4b20-90ac-e0f508a87d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:17.211040 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4cc6b0-22c3-4b20-90ac-e0f508a87d06 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:17.211040 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4cc6b0-22c3-4b20-90ac-e0f508a87d06 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:17.211040 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4cc6b0-22c3-4b20-90ac-e0f508a87d06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:17.228824 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4cc6b0-22c3-4b20-90ac-e0f508a87d06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:17.230276 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4cc6b0-22c3-4b20-90ac-e0f508a87d06 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:17.251408 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-cf4cc6b0-22c3-4b20-90ac-e0f508a87d06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:21:17.251408 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:21:17.251408 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:21:17.251408 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:21:17.251408 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:21:17.251408 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:21:17.251408 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:21:17.251408 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:21:17.251408 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:21:17.251408 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:21:17.251408 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:21:17.251408 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:21:17.251408 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:21:17.251408 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:21:17.251408 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:21:17.252827 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:21:17.252827 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:21:17.252827 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:21:17.252827 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:21:17.252827 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:21:17.252827 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:21:17.252827 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:21:17.252827 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:21:17.252827 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:21:17.252827 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:21:17.252827 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:21:17.252827 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:21:17.252827 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:21:17.252827 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:21:17.252827 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:21:17.252827 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:21:17.252827 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:21:17.253724 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:21:17.253724 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:21:17.253724 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:21:17.253724 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:21:17.253724 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 523/1029] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:21:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:21:17.261181 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6b06d4-a367-4a30-bbe6-d9b7588eaba5 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:17.326862 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:17.386923 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6b06d4-a367-4a30-bbe6-d9b7588eaba5 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:17.398128 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3799cc4f-b198-4cbe-ba69-6b5d2bdfdaf3 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:17.398437 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3799cc4f-b198-4cbe-ba69-6b5d2bdfdaf3 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:17.398704 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3799cc4f-b198-4cbe-ba69-6b5d2bdfdaf3 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:17.400258 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 524/1030] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:21:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 14:21:17.405689 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a462791b-7a6b-4607-a23f-facc3b81255d None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:17.405951 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a462791b-7a6b-4607-a23f-facc3b81255d None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:17.406239 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a462791b-7a6b-4607-a23f-facc3b81255d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:17.424020 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-a462791b-7a6b-4607-a23f-facc3b81255d None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:17.425419 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-a462791b-7a6b-4607-a23f-facc3b81255d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:17.443986 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216124b9-c061-4c84-a6e1-e6e806550eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:17.445128 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216124b9-c061-4c84-a6e1-e6e806550eaa None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:17.445574 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216124b9-c061-4c84-a6e1-e6e806550eaa None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:17.445856 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216124b9-c061-4c84-a6e1-e6e806550eaa None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:17.446954 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216124b9-c061-4c84-a6e1-e6e806550eaa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:17.447475 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216124b9-c061-4c84-a6e1-e6e806550eaa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:17.449493 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216124b9-c061-4c84-a6e1-e6e806550eaa None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:17.466330 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 507/1031] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jun 7 14:21:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:21:17.472991 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216124b9-c061-4c84-a6e1-e6e806550eaa None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:17.530616 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 525/1032] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:21:17 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 14:21:17.567369 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a462791b-7a6b-4607-a23f-facc3b81255d None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:17.626746 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:17.641850 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216124b9-c061-4c84-a6e1-e6e806550eaa None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:17.676642 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:17.704838 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f881fc-271c-43a2-a8f4-ea71407968c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:17.704838 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f881fc-271c-43a2-a8f4-ea71407968c0 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:17.704838 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f881fc-271c-43a2-a8f4-ea71407968c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:17.704838 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f881fc-271c-43a2-a8f4-ea71407968c0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:17.718462 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f881fc-271c-43a2-a8f4-ea71407968c0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:17.720817 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f881fc-271c-43a2-a8f4-ea71407968c0 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:17.731212 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a462791b-7a6b-4607-a23f-facc3b81255d None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:17.751605 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-f3f881fc-271c-43a2-a8f4-ea71407968c0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 07 14:21:17.751605 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:21:17.751605 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:21:17.751605 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:21:17.751605 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:21:17.751605 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:21:17.751605 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:21:17.751605 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:21:17.751605 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:21:17.751605 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:21:17.751605 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:21:17.751605 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:21:17.751605 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 14:21:17.751605 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 14:21:17.751605 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 14:21:17.751605 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 14:21:17.752695 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:21:17.752695 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:21:17.752695 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 14:21:17.752695 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:21:17.752695 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 14:21:17.752695 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:21:17.752695 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:21:17.752695 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:21:17.752695 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:21:17.752695 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:21:17.752695 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:21:17.752695 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:21:17.752695 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:21:17.752695 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:21:17.752695 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:21:17.752695 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:21:17.752695 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:21:17.752695 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:21:17.753659 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:21:17.753659 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:21:17.753659 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 14:21:17.753659 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 14:21:17.753659 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 14:21:17.753659 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 14:21:17.753659 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 14:21:17.753659 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 14:21:17.753659 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 07 14:21:17.753659 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:21:17.754220 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 508/1033] 10.222.0.85 () {56 vars in 1093 bytes} [Tue Jun 7 14:21:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:21:17.760149 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f881fc-271c-43a2-a8f4-ea71407968c0 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:17.778365 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d8c972-058c-430b-87e3-6723ad1c1e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:17.779171 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d8c972-058c-430b-87e3-6723ad1c1e6c None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:17.779511 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d8c972-058c-430b-87e3-6723ad1c1e6c None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:17.779791 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d8c972-058c-430b-87e3-6723ad1c1e6c None admin] PATH_INFO: `/v3/services/image` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:17.780490 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d8c972-058c-430b-87e3-6723ad1c1e6c None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:17.781892 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d8c972-058c-430b-87e3-6723ad1c1e6c None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:17.794908 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-e2d8c972-058c-430b-87e3-6723ad1c1e6c None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 07 14:21:17.794908 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:21:17.794908 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 14:21:17.794908 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 14:21:17.794908 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 14:21:17.794908 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 14:21:17.794908 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 14:21:17.794908 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:21:17.794908 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 07 14:21:17.794908 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:21:17.794908 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 14:21:17.794908 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:21:17.794908 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:21:17.794908 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:21:17.794908 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:21:17.794908 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:21:17.794908 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:21:17.796775 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:21:17.796775 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:21:17.796775 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:21:17.796775 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:21:17.796775 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:21:17.796775 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:21:17.796775 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 14:21:17.796775 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 14:21:17.796775 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 14:21:17.796775 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 14:21:17.796775 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:21:17.796775 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:21:17.796775 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:21:17.796775 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:21:17.796775 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:21:17.796775 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:21:17.798382 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:21:17.798382 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:21:17.798382 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:21:17.798382 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:21:17.798382 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:21:17.798382 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:21:17.798382 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:21:17.798382 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:21:17.798382 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:21:17.798382 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:21:17.798382 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 14:21:17.798382 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:21:17.798382 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 07 14:21:17.798382 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:21:17.798382 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 526/1034] 10.222.0.85 () {56 vars in 1105 bytes} [Tue Jun 7 14:21:17 2022] GET /identity/v3/services/image => generated 86 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:21:17.805112 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d8c972-058c-430b-87e3-6723ad1c1e6c None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:17.946895 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f881fc-271c-43a2-a8f4-ea71407968c0 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:17.967610 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d8c972-058c-430b-87e3-6723ad1c1e6c None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:17.978528 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:17.986986 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2534323-2b50-49ff-bb5e-f7275897a50a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:17.987668 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2534323-2b50-49ff-bb5e-f7275897a50a None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:17.987998 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2534323-2b50-49ff-bb5e-f7275897a50a None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:17.988333 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2534323-2b50-49ff-bb5e-f7275897a50a None admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:17.989105 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2534323-2b50-49ff-bb5e-f7275897a50a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:17.989412 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2534323-2b50-49ff-bb5e-f7275897a50a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:17.990883 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2534323-2b50-49ff-bb5e-f7275897a50a None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:18.014517 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 509/1035] 10.222.0.85 () {56 vars in 1133 bytes} [Tue Jun 7 14:21:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:21:18.015729 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2534323-2b50-49ff-bb5e-f7275897a50a None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:18.018752 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ff62c1-ef07-4fe0-8a55-fff2dc4c74cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:18.020472 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff62c1-ef07-4fe0-8a55-fff2dc4c74cd None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:18.020472 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff62c1-ef07-4fe0-8a55-fff2dc4c74cd None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:18.022220 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff62c1-ef07-4fe0-8a55-fff2dc4c74cd None admin] PATH_INFO: `/v3/services` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:18.022220 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ff62c1-ef07-4fe0-8a55-fff2dc4c74cd None admin] RBAC: Adding query filter params (name=image) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:18.022491 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ff62c1-ef07-4fe0-8a55-fff2dc4c74cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:18.023671 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ff62c1-ef07-4fe0-8a55-fff2dc4c74cd None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:18.029328 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 527/1036] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:21:17 2022] GET /identity/v3/services?name=image => generated 122 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:21:18.040595 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ff62c1-ef07-4fe0-8a55-fff2dc4c74cd None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:18.072951 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:18.145022 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2534323-2b50-49ff-bb5e-f7275897a50a None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:18.175732 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ff62c1-ef07-4fe0-8a55-fff2dc4c74cd None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:18.184773 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd20abfb-a2ab-43d0-a036-912bbaff4038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:18.185566 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd20abfb-a2ab-43d0-a036-912bbaff4038 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:18.185858 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd20abfb-a2ab-43d0-a036-912bbaff4038 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:18.186139 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd20abfb-a2ab-43d0-a036-912bbaff4038 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:18.212695 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37b3209-5acf-4dbf-bad6-97d623099025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:18.213375 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37b3209-5acf-4dbf-bad6-97d623099025 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:18.213661 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37b3209-5acf-4dbf-bad6-97d623099025 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:18.213938 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37b3209-5acf-4dbf-bad6-97d623099025 None admin] PATH_INFO: `/v3/services` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:18.214651 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37b3209-5acf-4dbf-bad6-97d623099025 None admin] RBAC: Adding query filter params (type=image) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:18.214967 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37b3209-5acf-4dbf-bad6-97d623099025 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:18.216355 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37b3209-5acf-4dbf-bad6-97d623099025 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:18.221840 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 528/1037] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:21:18 2022] GET /identity/v3/services?type=image => generated 350 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:21:18.227699 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37b3209-5acf-4dbf-bad6-97d623099025 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:18.269169 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:18.273579 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd20abfb-a2ab-43d0-a036-912bbaff4038 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:18.275740 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd20abfb-a2ab-43d0-a036-912bbaff4038 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:18.330448 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:18.373838 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37b3209-5acf-4dbf-bad6-97d623099025 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:18.409152 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 510/1038] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jun 7 14:21:18 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 07 14:21:18.414772 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76de9d2a-cdb3-400f-8be6-8303d9aa5bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:18.415454 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76de9d2a-cdb3-400f-8be6-8303d9aa5bf7 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:18.415743 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76de9d2a-cdb3-400f-8be6-8303d9aa5bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:18.416027 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76de9d2a-cdb3-400f-8be6-8303d9aa5bf7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:18.416290 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd20abfb-a2ab-43d0-a036-912bbaff4038 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:18.416885 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76de9d2a-cdb3-400f-8be6-8303d9aa5bf7 None admin] RBAC: Adding query filter params (interface=public, service_id=b02a648540c4490ab9c91d171db5d277, region_id=RegionOne) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:18.417204 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76de9d2a-cdb3-400f-8be6-8303d9aa5bf7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:18.418597 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76de9d2a-cdb3-400f-8be6-8303d9aa5bf7 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:18.424525 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 529/1039] 10.222.0.85 () {56 vars in 1251 bytes} [Tue Jun 7 14:21:18 2022] GET /identity/v3/endpoints?service_id=b02a648540c4490ab9c91d171db5d277&interface=public®ion_id=RegionOne => generated 194 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:21:18.430956 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76de9d2a-cdb3-400f-8be6-8303d9aa5bf7 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:18.548856 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:18.551193 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76de9d2a-cdb3-400f-8be6-8303d9aa5bf7 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:18.559485 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd20abfb-a2ab-43d0-a036-912bbaff4038 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:18.602960 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cd5b16-69a1-44ec-ac53-4cd0cbdffd5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:18.604233 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cd5b16-69a1-44ec-ac53-4cd0cbdffd5a None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:18.604679 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cd5b16-69a1-44ec-ac53-4cd0cbdffd5a None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:18.611067 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cd5b16-69a1-44ec-ac53-4cd0cbdffd5a None admin] PATH_INFO: `/v3/services` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:18.612125 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cd5b16-69a1-44ec-ac53-4cd0cbdffd5a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:18.613578 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cd5b16-69a1-44ec-ac53-4cd0cbdffd5a None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:18.614582 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29593bc5-f7e5-4dc6-a014-c963d9c46a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:18.616096 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29593bc5-f7e5-4dc6-a014-c963d9c46a41 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:18.616413 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29593bc5-f7e5-4dc6-a014-c963d9c46a41 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:18.616710 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29593bc5-f7e5-4dc6-a014-c963d9c46a41 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:18.627086 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 530/1040] 10.222.0.85 () {56 vars in 1087 bytes} [Tue Jun 7 14:21:18 2022] GET /identity/v3/services => generated 1957 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:21:18.631433 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:18.639619 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29593bc5-f7e5-4dc6-a014-c963d9c46a41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:18.641224 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29593bc5-f7e5-4dc6-a014-c963d9c46a41 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:18.665793 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-29593bc5-f7e5-4dc6-a014-c963d9c46a41 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:21:18.665793 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:21:18.665793 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:21:18.665793 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:21:18.665793 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:21:18.665793 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:21:18.665793 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:21:18.665793 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:21:18.665793 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:21:18.665793 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:21:18.665793 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:21:18.665793 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:21:18.665793 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:21:18.665793 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:21:18.665793 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:21:18.667020 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:21:18.667020 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:21:18.667020 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:21:18.667020 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:21:18.667020 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:21:18.667020 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:21:18.667020 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:21:18.667020 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:21:18.667020 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:21:18.667020 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:21:18.667020 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:21:18.667020 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:21:18.667020 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:21:18.667020 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:21:18.667020 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:21:18.667020 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:21:18.667020 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:21:18.674969 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:21:18.674969 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:21:18.674969 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:21:18.674969 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:21:18.674969 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 511/1041] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:21:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:21:18.677528 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cd5b16-69a1-44ec-ac53-4cd0cbdffd5a None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:18.809134 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cd5b16-69a1-44ec-ac53-4cd0cbdffd5a None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:18.865010 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6306b3a5-d636-432c-bdb7-e56679c1a088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:18.865688 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6306b3a5-d636-432c-bdb7-e56679c1a088 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:18.865971 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6306b3a5-d636-432c-bdb7-e56679c1a088 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:18.866248 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6306b3a5-d636-432c-bdb7-e56679c1a088 None admin] PATH_INFO: `/v3/domains` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:18.866975 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6306b3a5-d636-432c-bdb7-e56679c1a088 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:18.867281 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6306b3a5-d636-432c-bdb7-e56679c1a088 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:18.868718 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6306b3a5-d636-432c-bdb7-e56679c1a088 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:18.882595 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 531/1042] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jun 7 14:21:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:21:18.884583 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c812afc5-671d-41e8-b42d-59d0661df249 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:18.884873 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c812afc5-671d-41e8-b42d-59d0661df249 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:18.885146 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c812afc5-671d-41e8-b42d-59d0661df249 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:18.886630 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 512/1043] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:21:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:21:18.887871 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6306b3a5-d636-432c-bdb7-e56679c1a088 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:18.893360 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d933762-d78b-48d0-b204-9f999a598713 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:18.893673 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d933762-d78b-48d0-b204-9f999a598713 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:18.893983 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d933762-d78b-48d0-b204-9f999a598713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:18.912414 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-7d933762-d78b-48d0-b204-9f999a598713 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:18.913618 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-7d933762-d78b-48d0-b204-9f999a598713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:18.920833 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:18.993415 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 513/1044] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:21:18 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 14:21:19.009576 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6306b3a5-d636-432c-bdb7-e56679c1a088 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:19.017452 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d933762-d78b-48d0-b204-9f999a598713 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:19.048049 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c1b83f-cec8-44ad-8a20-48a7add4469c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:19.048987 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c1b83f-cec8-44ad-8a20-48a7add4469c None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:19.049393 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c1b83f-cec8-44ad-8a20-48a7add4469c None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:19.049670 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c1b83f-cec8-44ad-8a20-48a7add4469c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:19.059554 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c1b83f-cec8-44ad-8a20-48a7add4469c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:19.061170 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c1b83f-cec8-44ad-8a20-48a7add4469c None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:19.075644 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-68c1b83f-cec8-44ad-8a20-48a7add4469c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:21:19.075644 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:21:19.075644 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:21:19.075644 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:21:19.075644 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:21:19.075644 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:21:19.075644 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:21:19.075644 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:21:19.075644 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:21:19.075644 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:21:19.075644 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:21:19.075644 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:21:19.075644 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 14:21:19.075644 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 14:21:19.075644 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 14:21:19.076812 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 14:21:19.076812 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:21:19.076812 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:21:19.076812 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:21:19.076812 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:21:19.076812 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:21:19.076812 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:21:19.076812 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:21:19.076812 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:21:19.076812 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:21:19.076812 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:21:19.076812 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:21:19.076812 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:21:19.076812 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:21:19.076812 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:21:19.076812 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:21:19.076812 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:21:19.078159 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 14:21:19.078159 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 14:21:19.078159 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 14:21:19.078159 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 14:21:19.078159 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 14:21:19.078159 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 14:21:19.078159 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:21:19.078159 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:21:19.078159 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 532/1045] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jun 7 14:21:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:21:19.082400 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c1b83f-cec8-44ad-8a20-48a7add4469c None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:19.126944 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d933762-d78b-48d0-b204-9f999a598713 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:19.148935 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:19.163264 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df64fde-2839-4d10-ac3d-02f0028ef560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:19.164183 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df64fde-2839-4d10-ac3d-02f0028ef560 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:19.164494 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df64fde-2839-4d10-ac3d-02f0028ef560 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:19.164769 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df64fde-2839-4d10-ac3d-02f0028ef560 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:19.170430 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:19.175531 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df64fde-2839-4d10-ac3d-02f0028ef560 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:21:19.183359 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-5df64fde-2839-4d10-ac3d-02f0028ef560 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:21:19.183359 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:21:19.183359 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:21:19.183359 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:21:19.183359 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:21:19.183359 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:21:19.183359 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:21:19.183359 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:21:19.183359 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:21:19.183359 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:21:19.183359 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:21:19.183359 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:21:19.183359 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:21:19.183359 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:21:19.183359 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:21:19.183359 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:21:19.184345 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:21:19.184345 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:21:19.184345 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:21:19.184345 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:21:19.184345 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:21:19.184345 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:21:19.184345 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:21:19.184345 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:21:19.184345 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:21:19.184345 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:21:19.184345 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:21:19.184345 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:21:19.184345 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:21:19.184345 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:21:19.184345 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:21:19.184345 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:21:19.185774 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:21:19.185774 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:21:19.185774 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:21:19.185774 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:21:19.185774 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:21:19.185774 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:21:19.185774 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:21:19.185774 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:21:19.185774 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:21:19.185774 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:21:19.185774 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:21:19.185774 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:21:19.185774 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 514/1046] 10.222.0.85 () {56 vars in 1096 bytes} [Tue Jun 7 14:21:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:21:19.190331 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df64fde-2839-4d10-ac3d-02f0028ef560 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:19.195977 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c1b83f-cec8-44ad-8a20-48a7add4469c None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:19.232343 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa67c1e-b7f8-424f-893e-07e6909a37eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:19.233028 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa67c1e-b7f8-424f-893e-07e6909a37eb None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:19.233312 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa67c1e-b7f8-424f-893e-07e6909a37eb None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:19.233587 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa67c1e-b7f8-424f-893e-07e6909a37eb None admin] PATH_INFO: `/v3/projects` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:19.234489 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa67c1e-b7f8-424f-893e-07e6909a37eb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:19.234712 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa67c1e-b7f8-424f-893e-07e6909a37eb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:19.236125 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa67c1e-b7f8-424f-893e-07e6909a37eb None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:19.248973 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 533/1047] 10.222.0.85 () {56 vars in 1148 bytes} [Tue Jun 7 14:21:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:21:19.254697 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa67c1e-b7f8-424f-893e-07e6909a37eb None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:19.294072 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df64fde-2839-4d10-ac3d-02f0028ef560 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:19.330190 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1642b3-384f-49aa-9d2b-64547bd6ec48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:19.330778 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1642b3-384f-49aa-9d2b-64547bd6ec48 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:19.331067 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1642b3-384f-49aa-9d2b-64547bd6ec48 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:19.331347 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1642b3-384f-49aa-9d2b-64547bd6ec48 None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:19.332101 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1642b3-384f-49aa-9d2b-64547bd6ec48 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:19.332543 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1642b3-384f-49aa-9d2b-64547bd6ec48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:19.333930 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1642b3-384f-49aa-9d2b-64547bd6ec48 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:19.343842 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 515/1048] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jun 7 14:21:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:21:19.349282 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1642b3-384f-49aa-9d2b-64547bd6ec48 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:19.363295 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa67c1e-b7f8-424f-893e-07e6909a37eb None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:19.396077 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:19.403566 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02654410-a592-491c-b14c-f69cd5135251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:19.409726 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02654410-a592-491c-b14c-f69cd5135251 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:19.409726 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02654410-a592-491c-b14c-f69cd5135251 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:19.409726 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02654410-a592-491c-b14c-f69cd5135251 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:19.409726 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02654410-a592-491c-b14c-f69cd5135251 None admin] RBAC: Adding query filter params (role.id=5ca16aa9be5740d2902e27a98242edf4, scope.project.id=724785eb74e44daf8736fd01fc0d1839, user.id=8e857db4902e42989d0d040bbc8517c8) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:19.409726 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02654410-a592-491c-b14c-f69cd5135251 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:19.409726 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02654410-a592-491c-b14c-f69cd5135251 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:19.414749 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 534/1049] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jun 7 14:21:19 2022] GET /identity/v3/role_assignments?user.id=8e857db4902e42989d0d040bbc8517c8&scope.project.id=724785eb74e44daf8736fd01fc0d1839&role.id=5ca16aa9be5740d2902e27a98242edf4 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:21:19.469706 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1642b3-384f-49aa-9d2b-64547bd6ec48 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:19.519219 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d94c1af-15ee-4cb6-9ade-936e274f7aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:19.520124 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d94c1af-15ee-4cb6-9ade-936e274f7aff None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:19.520435 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d94c1af-15ee-4cb6-9ade-936e274f7aff None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:19.520720 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d94c1af-15ee-4cb6-9ade-936e274f7aff None admin] PATH_INFO: `/v3/users/58c51eb3059944b0b1dd8ebcf89650d0` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:19.536116 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d94c1af-15ee-4cb6-9ade-936e274f7aff None admin] RBAC: Authorizing `identity:get_user(user_id=58c51eb3059944b0b1dd8ebcf89650d0)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:19.537652 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d94c1af-15ee-4cb6-9ade-936e274f7aff None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:19.539634 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 516/1050] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jun 7 14:21:19 2022] GET /identity/v3/users/58c51eb3059944b0b1dd8ebcf89650d0 => generated 285 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:21:19.545396 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02654410-a592-491c-b14c-f69cd5135251 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:19.555599 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:19.662531 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02654410-a592-491c-b14c-f69cd5135251 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:19.697031 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f056ee-c5ed-4499-b746-766b44ab42ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:19.697922 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f056ee-c5ed-4499-b746-766b44ab42ee None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:19.698371 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f056ee-c5ed-4499-b746-766b44ab42ee None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:19.698798 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f056ee-c5ed-4499-b746-766b44ab42ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:19.785925 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f056ee-c5ed-4499-b746-766b44ab42ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:19.788206 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f056ee-c5ed-4499-b746-766b44ab42ee None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:19.799531 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:19.900904 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 535/1051] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jun 7 14:21:19 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 14:21:19.906752 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d94c1af-15ee-4cb6-9ade-936e274f7aff None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:20.026217 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:20.030934 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d94c1af-15ee-4cb6-9ade-936e274f7aff None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:20.075194 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e241b200-168e-4022-bcbf-094a4c370e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:20.076060 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e241b200-168e-4022-bcbf-094a4c370e49 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:20.076346 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e241b200-168e-4022-bcbf-094a4c370e49 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:20.076655 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e241b200-168e-4022-bcbf-094a4c370e49 None admin] PATH_INFO: `/v3/projects/8f2a8a9b8fce42f68f0ce7f1a365ae2e` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:20.077895 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e241b200-168e-4022-bcbf-094a4c370e49 None admin] RBAC: Authorizing `identity:get_project(project_id=8f2a8a9b8fce42f68f0ce7f1a365ae2e)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:20.079330 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e241b200-168e-4022-bcbf-094a4c370e49 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:20.084479 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 517/1052] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jun 7 14:21:19 2022] GET /identity/v3/projects/8f2a8a9b8fce42f68f0ce7f1a365ae2e => generated 311 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:21:20.087669 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f056ee-c5ed-4499-b746-766b44ab42ee None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:20.199210 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f056ee-c5ed-4499-b746-766b44ab42ee None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:20.221756 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:20.260766 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe29c5e-b7ce-444c-9a61-aa2f35290276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:20.261434 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe29c5e-b7ce-444c-9a61-aa2f35290276 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:20.261720 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe29c5e-b7ce-444c-9a61-aa2f35290276 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:20.262004 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe29c5e-b7ce-444c-9a61-aa2f35290276 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:20.262848 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe29c5e-b7ce-444c-9a61-aa2f35290276 None admin] RBAC: Adding query filter params (role.id=9baff2b18f2649f7b757ff2e1f63c853, scope.project.id=8f2a8a9b8fce42f68f0ce7f1a365ae2e, user.id=58c51eb3059944b0b1dd8ebcf89650d0) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:20.263165 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe29c5e-b7ce-444c-9a61-aa2f35290276 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:20.264671 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe29c5e-b7ce-444c-9a61-aa2f35290276 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:20.271636 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 536/1053] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jun 7 14:21:20 2022] GET /identity/v3/role_assignments?user.id=58c51eb3059944b0b1dd8ebcf89650d0&scope.project.id=8f2a8a9b8fce42f68f0ce7f1a365ae2e&role.id=9baff2b18f2649f7b757ff2e1f63c853 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:21:20.975596 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c0b4b7-e9e9-4c7c-9440-8931bf50900a None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:20.976034 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c0b4b7-e9e9-4c7c-9440-8931bf50900a None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:20.976318 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c0b4b7-e9e9-4c7c-9440-8931bf50900a None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:20.977717 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 518/1054] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:21:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:21:20.983079 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d9bede-c92d-42dd-bf45-9bbb1a0648c7 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:20.983527 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d9bede-c92d-42dd-bf45-9bbb1a0648c7 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:20.983617 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d9bede-c92d-42dd-bf45-9bbb1a0648c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:21.002742 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-a7d9bede-c92d-42dd-bf45-9bbb1a0648c7 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:21.003905 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-a7d9bede-c92d-42dd-bf45-9bbb1a0648c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:21.098592 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 537/1055] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:21:20 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:21:21.123555 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c0b4b7-e9e9-4c7c-9440-8931bf50900a None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:21.262113 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c0b4b7-e9e9-4c7c-9440-8931bf50900a None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:21.306642 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9b6a3c-1be5-4203-9b51-95e382856628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:21.307474 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9b6a3c-1be5-4203-9b51-95e382856628 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:21.307727 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9b6a3c-1be5-4203-9b51-95e382856628 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:21.308003 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9b6a3c-1be5-4203-9b51-95e382856628 None admin] PATH_INFO: `/v3/services/image` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:21.308707 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9b6a3c-1be5-4203-9b51-95e382856628 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:21.310097 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9b6a3c-1be5-4203-9b51-95e382856628 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:21.323050 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-dc9b6a3c-1be5-4203-9b51-95e382856628 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 07 14:21:21.323050 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:21:21.323050 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 14:21:21.323050 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 14:21:21.323050 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 14:21:21.323050 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 14:21:21.323050 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 14:21:21.323050 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:21:21.323050 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 07 14:21:21.323050 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:21:21.323050 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 14:21:21.323050 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:21:21.323050 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:21:21.323050 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:21:21.323050 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:21:21.323050 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:21:21.323050 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:21:21.324219 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:21:21.324219 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:21:21.324219 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:21:21.324219 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:21:21.324219 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:21:21.324219 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:21:21.324219 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 14:21:21.324219 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 14:21:21.324219 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 14:21:21.324219 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 14:21:21.324219 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:21:21.324219 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:21:21.324219 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:21:21.324219 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:21:21.324219 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:21:21.324219 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:21:21.325621 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:21:21.325621 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:21:21.325621 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:21:21.325621 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:21:21.325621 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:21:21.325621 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:21:21.325621 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:21:21.325621 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:21:21.325621 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:21:21.325621 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:21:21.325621 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 14:21:21.325621 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:21:21.325621 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 07 14:21:21.325621 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:21:21.328753 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 519/1056] 10.222.0.85 () {56 vars in 1105 bytes} [Tue Jun 7 14:21:21 2022] GET /identity/v3/services/image => generated 86 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:21:21.330491 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d9bede-c92d-42dd-bf45-9bbb1a0648c7 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:21.456124 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:21.461224 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d9bede-c92d-42dd-bf45-9bbb1a0648c7 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:21.499262 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9f1e99-f5e3-4018-9f30-4babe90a4ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:21.499959 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9f1e99-f5e3-4018-9f30-4babe90a4ecb None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:21.500245 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9f1e99-f5e3-4018-9f30-4babe90a4ecb None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:21.500582 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9f1e99-f5e3-4018-9f30-4babe90a4ecb None admin] PATH_INFO: `/v3/services` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:21.501336 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9f1e99-f5e3-4018-9f30-4babe90a4ecb None admin] RBAC: Adding query filter params (name=image) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:21.501649 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9f1e99-f5e3-4018-9f30-4babe90a4ecb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:21.503155 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9f1e99-f5e3-4018-9f30-4babe90a4ecb None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:21.515623 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 538/1057] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:21:21 2022] GET /identity/v3/services?name=image => generated 122 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:21:21.517776 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9b6a3c-1be5-4203-9b51-95e382856628 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:21.551926 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:21.656660 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9b6a3c-1be5-4203-9b51-95e382856628 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:21.695355 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d63b4c-46b2-40be-91e9-36e2b8c2bbbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:21.696025 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d63b4c-46b2-40be-91e9-36e2b8c2bbbe None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:21.696311 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d63b4c-46b2-40be-91e9-36e2b8c2bbbe None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:21.696827 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d63b4c-46b2-40be-91e9-36e2b8c2bbbe None admin] PATH_INFO: `/v3/services` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:21.697552 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d63b4c-46b2-40be-91e9-36e2b8c2bbbe None admin] RBAC: Adding query filter params (type=image) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:21.697861 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d63b4c-46b2-40be-91e9-36e2b8c2bbbe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:21.699217 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d63b4c-46b2-40be-91e9-36e2b8c2bbbe None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:21.704574 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 520/1058] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:21:21 2022] GET /identity/v3/services?type=image => generated 350 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:21:21.711711 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9f1e99-f5e3-4018-9f30-4babe90a4ecb None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:21.764258 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1b1e35-466b-49e4-849a-45a17991502c None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:21.764258 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1b1e35-466b-49e4-849a-45a17991502c None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:21.764807 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1b1e35-466b-49e4-849a-45a17991502c None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:21.766012 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 521/1059] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:21:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 14:21:21.770660 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af0ec91-2c7f-4b1f-b4bd-ded70a951232 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:21.770981 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af0ec91-2c7f-4b1f-b4bd-ded70a951232 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:21.771254 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af0ec91-2c7f-4b1f-b4bd-ded70a951232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:21.790663 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-2af0ec91-2c7f-4b1f-b4bd-ded70a951232 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:21.791976 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-2af0ec91-2c7f-4b1f-b4bd-ded70a951232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:21.840239 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9f1e99-f5e3-4018-9f30-4babe90a4ecb None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:21.861103 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 522/1060] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:21:21 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 14:21:21.881906 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6a9519-1881-4069-bce4-1706ba7366e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:21.882850 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6a9519-1881-4069-bce4-1706ba7366e9 None admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:21.883306 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6a9519-1881-4069-bce4-1706ba7366e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:21.883745 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6a9519-1881-4069-bce4-1706ba7366e9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:21.884697 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6a9519-1881-4069-bce4-1706ba7366e9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:21.885172 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af0ec91-2c7f-4b1f-b4bd-ded70a951232 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:21.886411 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6a9519-1881-4069-bce4-1706ba7366e9 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:21.911306 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:21.980888 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 539/1061] 10.222.0.85 () {60 vars in 1144 bytes} [Tue Jun 7 14:21:21 2022] POST /identity/v3/endpoints => generated 327 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:21:21.989045 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af0ec91-2c7f-4b1f-b4bd-ded70a951232 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:22.038516 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6f3b27-869b-4323-914e-fb9630d1bff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:22.039248 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6f3b27-869b-4323-914e-fb9630d1bff2 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:22.039534 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6f3b27-869b-4323-914e-fb9630d1bff2 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:22.039815 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6f3b27-869b-4323-914e-fb9630d1bff2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:22.052580 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6f3b27-869b-4323-914e-fb9630d1bff2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:21:22.060854 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-2f6f3b27-869b-4323-914e-fb9630d1bff2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 14:21:22.060854 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:21:22.060854 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:21:22.060854 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:21:22.060854 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:21:22.060854 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:21:22.060854 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:21:22.060854 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:21:22.060854 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:21:22.060854 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:21:22.060854 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:21:22.060854 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:21:22.060854 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:21:22.060854 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:21:22.060854 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:21:22.060854 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:21:22.061890 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:21:22.061890 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:21:22.061890 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:21:22.061890 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:21:22.061890 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:21:22.061890 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:21:22.061890 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:21:22.061890 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:21:22.061890 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:21:22.061890 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:21:22.061890 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:21:22.061890 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:21:22.061890 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:21:22.061890 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:21:22.061890 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:21:22.061890 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:21:22.062777 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:21:22.062777 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:21:22.062777 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:21:22.062777 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:21:22.062777 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:21:22.062777 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:21:22.062777 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:21:22.062777 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:21:22.062777 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:21:22.062777 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:21:22.062777 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 14:21:22.062777 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:21:22.062777 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 523/1062] 10.222.0.85 () {56 vars in 1120 bytes} [Tue Jun 7 14:21:21 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:21:22.069082 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6a9519-1881-4069-bce4-1706ba7366e9 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:22.097138 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:22.183137 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6a9519-1881-4069-bce4-1706ba7366e9 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:22.224399 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b7364f-341c-486b-afd6-d07b7ddde60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:22.225116 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b7364f-341c-486b-afd6-d07b7ddde60d None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:22.225403 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b7364f-341c-486b-afd6-d07b7ddde60d None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:22.225676 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b7364f-341c-486b-afd6-d07b7ddde60d None admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:22.226426 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b7364f-341c-486b-afd6-d07b7ddde60d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:22.226735 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b7364f-341c-486b-afd6-d07b7ddde60d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:22.228121 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b7364f-341c-486b-afd6-d07b7ddde60d None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:22.237360 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 540/1063] 10.222.0.85 () {56 vars in 1115 bytes} [Tue Jun 7 14:21:22 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:21:22.243836 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6f3b27-869b-4323-914e-fb9630d1bff2 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:22.334879 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:22.364194 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6f3b27-869b-4323-914e-fb9630d1bff2 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:22.408695 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1cb514-74f5-4caa-b559-8c4857818e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:22.409486 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1cb514-74f5-4caa-b559-8c4857818e61 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:22.409771 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1cb514-74f5-4caa-b559-8c4857818e61 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:22.410044 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1cb514-74f5-4caa-b559-8c4857818e61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:22.421688 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1cb514-74f5-4caa-b559-8c4857818e61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:22.423348 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1cb514-74f5-4caa-b559-8c4857818e61 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:22.447859 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-8e1cb514-74f5-4caa-b559-8c4857818e61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:21:22.447859 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:21:22.447859 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:21:22.447859 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:21:22.447859 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:21:22.447859 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:21:22.447859 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:21:22.447859 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:21:22.447859 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:21:22.447859 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:21:22.447859 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:21:22.447859 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:21:22.447859 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:21:22.447859 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:21:22.447859 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:21:22.448859 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:21:22.448859 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:21:22.448859 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:21:22.448859 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:21:22.448859 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:21:22.448859 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:21:22.448859 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:21:22.448859 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:21:22.448859 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:21:22.448859 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:21:22.448859 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:21:22.448859 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:21:22.448859 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:21:22.448859 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:21:22.448859 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:21:22.448859 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:21:22.448859 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:21:22.449758 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:21:22.449758 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:21:22.449758 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:21:22.449758 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:21:22.452506 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 524/1064] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:21:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:21:22.455655 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b7364f-341c-486b-afd6-d07b7ddde60d None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:22.548328 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:22.568780 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b7364f-341c-486b-afd6-d07b7ddde60d None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:22.608993 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b05889-5429-4251-bd3b-dc4bb36c4c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:22.609581 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b05889-5429-4251-bd3b-dc4bb36c4c05 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:22.609862 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b05889-5429-4251-bd3b-dc4bb36c4c05 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:22.610137 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b05889-5429-4251-bd3b-dc4bb36c4c05 None admin] PATH_INFO: `/v3/domains` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:22.610857 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b05889-5429-4251-bd3b-dc4bb36c4c05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:22.611162 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b05889-5429-4251-bd3b-dc4bb36c4c05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:22.612560 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b05889-5429-4251-bd3b-dc4bb36c4c05 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:22.625829 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 541/1065] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jun 7 14:21:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:21:22.632606 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1cb514-74f5-4caa-b559-8c4857818e61 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:22.688104 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf8e0ba-b435-486c-8c50-aac384839c2c None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:22.688676 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf8e0ba-b435-486c-8c50-aac384839c2c None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:22.689171 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf8e0ba-b435-486c-8c50-aac384839c2c None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:22.691707 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 542/1066] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:21:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:21:22.697071 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8795dd97-a0df-49d5-ab90-3fd45b06d8bb None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:22.697363 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8795dd97-a0df-49d5-ab90-3fd45b06d8bb None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:22.697703 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8795dd97-a0df-49d5-ab90-3fd45b06d8bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:22.721225 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-8795dd97-a0df-49d5-ab90-3fd45b06d8bb None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:22.722442 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-8795dd97-a0df-49d5-ab90-3fd45b06d8bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:22.758394 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1cb514-74f5-4caa-b559-8c4857818e61 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:22.798710 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 543/1067] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:21:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 14:21:22.802692 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe613b33-3de6-4511-aee3-785ec05f6958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:22.803502 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe613b33-3de6-4511-aee3-785ec05f6958 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:22.803787 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe613b33-3de6-4511-aee3-785ec05f6958 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:22.804063 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe613b33-3de6-4511-aee3-785ec05f6958 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:22.806116 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:22.813086 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe613b33-3de6-4511-aee3-785ec05f6958 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:22.814601 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe613b33-3de6-4511-aee3-785ec05f6958 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:22.828642 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-fe613b33-3de6-4511-aee3-785ec05f6958 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 07 14:21:22.828642 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:21:22.828642 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:21:22.828642 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:21:22.828642 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:21:22.828642 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:21:22.828642 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:21:22.828642 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:21:22.828642 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:21:22.828642 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:21:22.828642 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:21:22.828642 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:21:22.828642 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 14:21:22.828642 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 14:21:22.828642 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 14:21:22.828642 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 14:21:22.830458 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:21:22.830458 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:21:22.830458 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 14:21:22.830458 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:21:22.830458 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 14:21:22.830458 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:21:22.830458 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:21:22.830458 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:21:22.830458 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:21:22.830458 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:21:22.830458 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:21:22.830458 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:21:22.830458 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:21:22.830458 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:21:22.830458 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:21:22.830458 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:21:22.830458 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:21:22.830458 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:21:22.832037 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:21:22.832037 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:21:22.832037 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 14:21:22.832037 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 14:21:22.832037 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 14:21:22.832037 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 14:21:22.832037 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 14:21:22.832037 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 14:21:22.832037 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 07 14:21:22.832037 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:21:22.832962 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 525/1068] 10.222.0.85 () {56 vars in 1093 bytes} [Tue Jun 7 14:21:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:21:22.835336 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8795dd97-a0df-49d5-ab90-3fd45b06d8bb None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:22.837723 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe613b33-3de6-4511-aee3-785ec05f6958 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:22.917504 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:22.950043 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8795dd97-a0df-49d5-ab90-3fd45b06d8bb None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:22.965555 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe613b33-3de6-4511-aee3-785ec05f6958 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:22.986233 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37777dd-cacf-4ee8-89c1-703b1cd398ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:22.987326 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37777dd-cacf-4ee8-89c1-703b1cd398ab None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:22.987812 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37777dd-cacf-4ee8-89c1-703b1cd398ab None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:22.988084 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37777dd-cacf-4ee8-89c1-703b1cd398ab None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:22.999691 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37777dd-cacf-4ee8-89c1-703b1cd398ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:21:23.005734 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586c730a-24e8-4c47-96ad-8c705f421817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:23.006406 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586c730a-24e8-4c47-96ad-8c705f421817 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:23.006699 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586c730a-24e8-4c47-96ad-8c705f421817 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:23.006982 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586c730a-24e8-4c47-96ad-8c705f421817 None admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:23.007823 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586c730a-24e8-4c47-96ad-8c705f421817 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:23.007986 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-c37777dd-cacf-4ee8-89c1-703b1cd398ab None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:21:23.007986 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:21:23.007986 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:21:23.007986 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:21:23.007986 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:21:23.007986 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:21:23.007986 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:21:23.007986 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:21:23.007986 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:21:23.007986 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:21:23.007986 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:21:23.007986 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:21:23.007986 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:21:23.007986 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:21:23.007986 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:21:23.007986 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:21:23.008918 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:21:23.008918 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:21:23.008918 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:21:23.008918 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:21:23.008918 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:21:23.008918 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:21:23.008918 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:21:23.008918 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:21:23.008918 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:21:23.008918 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:21:23.008918 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:21:23.008918 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:21:23.008918 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:21:23.008918 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:21:23.008918 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:21:23.008918 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:21:23.009747 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:21:23.009747 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:21:23.009747 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:21:23.009747 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:21:23.009747 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:21:23.009747 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:21:23.009747 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:21:23.009747 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:21:23.009747 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:21:23.009747 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:21:23.009747 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:21:23.009747 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:21:23.010916 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586c730a-24e8-4c47-96ad-8c705f421817 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:23.010916 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586c730a-24e8-4c47-96ad-8c705f421817 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:23.014093 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 544/1069] 10.222.0.85 () {56 vars in 1096 bytes} [Tue Jun 7 14:21:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:21:23.014956 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37777dd-cacf-4ee8-89c1-703b1cd398ab None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:23.028192 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 526/1070] 10.222.0.85 () {56 vars in 1133 bytes} [Tue Jun 7 14:21:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:21:23.033409 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586c730a-24e8-4c47-96ad-8c705f421817 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:23.094566 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:23.129910 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37777dd-cacf-4ee8-89c1-703b1cd398ab None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:23.145432 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586c730a-24e8-4c47-96ad-8c705f421817 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:23.162555 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:23.168330 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4a47a6-03e9-4dd6-b8fa-88d7b40767e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:23.168967 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4a47a6-03e9-4dd6-b8fa-88d7b40767e9 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:23.169256 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4a47a6-03e9-4dd6-b8fa-88d7b40767e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:23.169536 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4a47a6-03e9-4dd6-b8fa-88d7b40767e9 None admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:23.170412 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4a47a6-03e9-4dd6-b8fa-88d7b40767e9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:23.170865 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4a47a6-03e9-4dd6-b8fa-88d7b40767e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:23.172275 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4a47a6-03e9-4dd6-b8fa-88d7b40767e9 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:23.182177 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 545/1071] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jun 7 14:21:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:21:23.187679 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4a47a6-03e9-4dd6-b8fa-88d7b40767e9 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:23.188688 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ddf467-5ba5-4c22-9f27-ab06bc9a3e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:23.189494 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ddf467-5ba5-4c22-9f27-ab06bc9a3e7a None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:23.189783 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ddf467-5ba5-4c22-9f27-ab06bc9a3e7a None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:23.190143 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ddf467-5ba5-4c22-9f27-ab06bc9a3e7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:23.201290 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ddf467-5ba5-4c22-9f27-ab06bc9a3e7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:23.202780 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ddf467-5ba5-4c22-9f27-ab06bc9a3e7a None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:23.221523 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-32ddf467-5ba5-4c22-9f27-ab06bc9a3e7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:21:23.221523 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:21:23.221523 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:21:23.221523 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:21:23.221523 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:21:23.221523 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:21:23.221523 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:21:23.221523 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:21:23.221523 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:21:23.221523 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:21:23.221523 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:21:23.221523 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:21:23.221523 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:21:23.221523 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:21:23.221523 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:21:23.231593 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:21:23.231593 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:21:23.231593 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:21:23.231593 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:21:23.231593 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:21:23.231593 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:21:23.231593 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:21:23.231593 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:21:23.231593 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:21:23.231593 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:21:23.231593 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:21:23.231593 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:21:23.231593 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:21:23.231593 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:21:23.231593 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:21:23.231593 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:21:23.231593 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:21:23.232528 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:21:23.232528 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:21:23.232528 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:21:23.232528 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:21:23.232528 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 527/1072] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:21:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:21:23.232528 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ddf467-5ba5-4c22-9f27-ab06bc9a3e7a None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:23.301821 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4a47a6-03e9-4dd6-b8fa-88d7b40767e9 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:23.340395 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b9044e-888e-40fe-843a-d54c63f80e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:23.341289 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b9044e-888e-40fe-843a-d54c63f80e14 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:23.341571 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b9044e-888e-40fe-843a-d54c63f80e14 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:23.341847 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b9044e-888e-40fe-843a-d54c63f80e14 None admin] PATH_INFO: `/v3/users/58c51eb3059944b0b1dd8ebcf89650d0` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:23.343041 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b9044e-888e-40fe-843a-d54c63f80e14 None admin] RBAC: Authorizing `identity:get_user(user_id=58c51eb3059944b0b1dd8ebcf89650d0)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:23.344401 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ddf467-5ba5-4c22-9f27-ab06bc9a3e7a None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:23.344838 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b9044e-888e-40fe-843a-d54c63f80e14 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:23.352348 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 546/1073] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jun 7 14:21:23 2022] GET /identity/v3/users/58c51eb3059944b0b1dd8ebcf89650d0 => generated 285 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:21:23.352548 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:23.352646 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b9044e-888e-40fe-843a-d54c63f80e14 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:23.389440 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb483036-29bc-4949-95ef-8724ecb362ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:23.390127 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb483036-29bc-4949-95ef-8724ecb362ef None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:23.390413 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb483036-29bc-4949-95ef-8724ecb362ef None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:23.391285 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb483036-29bc-4949-95ef-8724ecb362ef None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:23.391447 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb483036-29bc-4949-95ef-8724ecb362ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:23.391697 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb483036-29bc-4949-95ef-8724ecb362ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:23.393081 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb483036-29bc-4949-95ef-8724ecb362ef None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:23.405447 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 528/1074] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jun 7 14:21:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:21:23.410794 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb483036-29bc-4949-95ef-8724ecb362ef None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:23.428492 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:23.469439 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b9044e-888e-40fe-843a-d54c63f80e14 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:23.509585 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b96145-8593-4eb6-9096-3964d491fc98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:23.510443 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b96145-8593-4eb6-9096-3964d491fc98 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:23.510722 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b96145-8593-4eb6-9096-3964d491fc98 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:23.510991 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b96145-8593-4eb6-9096-3964d491fc98 None admin] PATH_INFO: `/v3/projects/8f2a8a9b8fce42f68f0ce7f1a365ae2e` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:23.512157 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b96145-8593-4eb6-9096-3964d491fc98 None admin] RBAC: Authorizing `identity:get_project(project_id=8f2a8a9b8fce42f68f0ce7f1a365ae2e)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:23.513616 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b96145-8593-4eb6-9096-3964d491fc98 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:23.520050 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 547/1075] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jun 7 14:21:23 2022] GET /identity/v3/projects/8f2a8a9b8fce42f68f0ce7f1a365ae2e => generated 311 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:21:23.521169 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b96145-8593-4eb6-9096-3964d491fc98 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:23.522258 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb483036-29bc-4949-95ef-8724ecb362ef None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:23.559612 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b4a11b-7139-4193-81e2-82ad0eeaeb18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:23.560432 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b4a11b-7139-4193-81e2-82ad0eeaeb18 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:23.560723 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b4a11b-7139-4193-81e2-82ad0eeaeb18 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:23.561004 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b4a11b-7139-4193-81e2-82ad0eeaeb18 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:23.568468 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b4a11b-7139-4193-81e2-82ad0eeaeb18 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:23.569982 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b4a11b-7139-4193-81e2-82ad0eeaeb18 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:23.584343 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-70b4a11b-7139-4193-81e2-82ad0eeaeb18 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:21:23.584343 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:21:23.584343 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:21:23.584343 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:21:23.584343 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:21:23.584343 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:21:23.584343 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:21:23.584343 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:21:23.584343 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:21:23.584343 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:21:23.584343 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:21:23.584343 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:21:23.584343 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 14:21:23.584343 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 14:21:23.584343 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 14:21:23.585445 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 14:21:23.585445 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:21:23.585445 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:21:23.585445 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:21:23.585445 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:21:23.585445 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:21:23.585445 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:21:23.585445 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:21:23.585445 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:21:23.585445 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:21:23.585445 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:21:23.585445 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:21:23.585445 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:21:23.585445 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:21:23.585445 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:21:23.585445 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:21:23.585445 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:21:23.586465 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 14:21:23.586465 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 14:21:23.586465 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 14:21:23.586465 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 14:21:23.586465 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 14:21:23.586465 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 14:21:23.586465 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:21:23.586465 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:21:23.586465 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 529/1076] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jun 7 14:21:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:21:23.591141 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b4a11b-7139-4193-81e2-82ad0eeaeb18 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:23.619815 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:23.636415 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b96145-8593-4eb6-9096-3964d491fc98 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:23.661010 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:23.669176 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f917430-c12e-4583-9c83-1c1a487c6a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:23.670204 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f917430-c12e-4583-9c83-1c1a487c6a9d None admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:23.671324 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f917430-c12e-4583-9c83-1c1a487c6a9d None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:23.671324 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f917430-c12e-4583-9c83-1c1a487c6a9d None admin] PATH_INFO: `/v3/projects/8f2a8a9b8fce42f68f0ce7f1a365ae2e/users/58c51eb3059944b0b1dd8ebcf89650d0/roles/9baff2b18f2649f7b757ff2e1f63c853` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:23.673178 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f917430-c12e-4583-9c83-1c1a487c6a9d None admin] RBAC: Authorizing `identity:create_grant(project_id=8f2a8a9b8fce42f68f0ce7f1a365ae2e, user_id=58c51eb3059944b0b1dd8ebcf89650d0, role_id=9baff2b18f2649f7b757ff2e1f63c853)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:23.674690 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f917430-c12e-4583-9c83-1c1a487c6a9d None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:23.706833 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b4a11b-7139-4193-81e2-82ad0eeaeb18 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:23.743472 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b77a9f9-8488-44ea-ba71-9506e8c1cb63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:23.744730 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b77a9f9-8488-44ea-ba71-9506e8c1cb63 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:23.744730 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b77a9f9-8488-44ea-ba71-9506e8c1cb63 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:23.744730 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b77a9f9-8488-44ea-ba71-9506e8c1cb63 None admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:23.746181 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b77a9f9-8488-44ea-ba71-9506e8c1cb63 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:23.746181 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b77a9f9-8488-44ea-ba71-9506e8c1cb63 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:23.747426 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b77a9f9-8488-44ea-ba71-9506e8c1cb63 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:23.763533 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 530/1077] 10.222.0.85 () {56 vars in 1148 bytes} [Tue Jun 7 14:21:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:21:23.772536 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b77a9f9-8488-44ea-ba71-9506e8c1cb63 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:23.779220 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 548/1078] 10.222.0.85 () {58 vars in 1439 bytes} [Tue Jun 7 14:21:23 2022] PUT /identity/v3/projects/8f2a8a9b8fce42f68f0ce7f1a365ae2e/users/58c51eb3059944b0b1dd8ebcf89650d0/roles/9baff2b18f2649f7b757ff2e1f63c853 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:21:23.896449 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:23.913919 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b77a9f9-8488-44ea-ba71-9506e8c1cb63 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:23.961807 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bfee60-d391-44f1-80d6-e1f288cb19eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:23.962733 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bfee60-d391-44f1-80d6-e1f288cb19eb None admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:23.963020 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bfee60-d391-44f1-80d6-e1f288cb19eb None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:23.963300 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bfee60-d391-44f1-80d6-e1f288cb19eb None admin] PATH_INFO: `/v3/projects/724785eb74e44daf8736fd01fc0d1839/users/8e857db4902e42989d0d040bbc8517c8/roles/5ca16aa9be5740d2902e27a98242edf4` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:23.965681 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bfee60-d391-44f1-80d6-e1f288cb19eb None admin] RBAC: Authorizing `identity:create_grant(project_id=724785eb74e44daf8736fd01fc0d1839, user_id=8e857db4902e42989d0d040bbc8517c8, role_id=5ca16aa9be5740d2902e27a98242edf4)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:23.967201 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bfee60-d391-44f1-80d6-e1f288cb19eb None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:24.035158 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 531/1079] 10.222.0.85 () {58 vars in 1439 bytes} [Tue Jun 7 14:21:23 2022] PUT /identity/v3/projects/724785eb74e44daf8736fd01fc0d1839/users/8e857db4902e42989d0d040bbc8517c8/roles/5ca16aa9be5740d2902e27a98242edf4 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:21:24.316017 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18275874-b877-4838-a8da-8998d9f59f67 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:24.316557 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18275874-b877-4838-a8da-8998d9f59f67 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:24.316557 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18275874-b877-4838-a8da-8998d9f59f67 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:24.318692 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 549/1080] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:21:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:21:24.324438 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f59750-1ceb-4e20-8ac3-93a16f54e838 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:24.324733 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f59750-1ceb-4e20-8ac3-93a16f54e838 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:24.325012 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f59750-1ceb-4e20-8ac3-93a16f54e838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:24.342637 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-b2f59750-1ceb-4e20-8ac3-93a16f54e838 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:24.343834 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-b2f59750-1ceb-4e20-8ac3-93a16f54e838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:24.422913 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 532/1081] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:21:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 14:21:24.445714 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18275874-b877-4838-a8da-8998d9f59f67 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:24.551318 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18275874-b877-4838-a8da-8998d9f59f67 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:24.588077 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2db4d88-1d07-4dd3-a97a-fe1b490667f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:24.588678 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2db4d88-1d07-4dd3-a97a-fe1b490667f6 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:24.588962 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2db4d88-1d07-4dd3-a97a-fe1b490667f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:24.589238 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2db4d88-1d07-4dd3-a97a-fe1b490667f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:24.651086 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:24.662119 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2db4d88-1d07-4dd3-a97a-fe1b490667f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:24.673285 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2db4d88-1d07-4dd3-a97a-fe1b490667f6 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:24.766575 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 550/1082] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jun 7 14:21:24 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 14:21:24.773259 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f59750-1ceb-4e20-8ac3-93a16f54e838 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:24.848158 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:24.879126 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f59750-1ceb-4e20-8ac3-93a16f54e838 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:24.917535 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec40d75-60a9-4ad4-a37c-92c18a57f817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:24.918642 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec40d75-60a9-4ad4-a37c-92c18a57f817 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:24.919108 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec40d75-60a9-4ad4-a37c-92c18a57f817 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:24.919582 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec40d75-60a9-4ad4-a37c-92c18a57f817 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:24.931200 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec40d75-60a9-4ad4-a37c-92c18a57f817 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:24.932935 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec40d75-60a9-4ad4-a37c-92c18a57f817 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:24.953265 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-6ec40d75-60a9-4ad4-a37c-92c18a57f817 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:21:24.953265 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:21:24.953265 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:21:24.953265 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:21:24.953265 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:21:24.953265 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:21:24.953265 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:21:24.953265 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:21:24.953265 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:21:24.953265 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:21:24.953265 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:21:24.953265 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:21:24.953265 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:21:24.953265 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:21:24.953265 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:21:24.954209 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:21:24.954209 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:21:24.954209 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:21:24.954209 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:21:24.954209 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:21:24.954209 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:21:24.954209 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:21:24.954209 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:21:24.954209 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:21:24.954209 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:21:24.954209 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:21:24.954209 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:21:24.954209 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:21:24.954209 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:21:24.954209 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:21:24.954209 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:21:24.954209 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:21:24.955115 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:21:24.955115 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:21:24.955115 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:21:24.955115 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:21:24.960855 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 533/1083] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:21:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:21:24.965042 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2db4d88-1d07-4dd3-a97a-fe1b490667f6 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:25.057336 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:25.068063 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2db4d88-1d07-4dd3-a97a-fe1b490667f6 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:25.106100 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe58f50-2c63-42a4-9809-ac05c7c55201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:25.106684 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe58f50-2c63-42a4-9809-ac05c7c55201 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:25.106966 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe58f50-2c63-42a4-9809-ac05c7c55201 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:25.107238 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe58f50-2c63-42a4-9809-ac05c7c55201 None admin] PATH_INFO: `/v3/domains` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:25.107957 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe58f50-2c63-42a4-9809-ac05c7c55201 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:25.108262 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe58f50-2c63-42a4-9809-ac05c7c55201 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:25.109649 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe58f50-2c63-42a4-9809-ac05c7c55201 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:25.128403 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 551/1084] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jun 7 14:21:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:21:26.046358 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f910d6a-0c55-459d-86e6-51b10e1814c5 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:26.046911 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f910d6a-0c55-459d-86e6-51b10e1814c5 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:26.047007 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f910d6a-0c55-459d-86e6-51b10e1814c5 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:26.048361 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 534/1085] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:21:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:21:26.053696 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29748556-9674-4eec-85d8-d39700cdca86 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:26.054131 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29748556-9674-4eec-85d8-d39700cdca86 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:26.054298 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29748556-9674-4eec-85d8-d39700cdca86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:26.070784 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-29748556-9674-4eec-85d8-d39700cdca86 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:26.071959 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-29748556-9674-4eec-85d8-d39700cdca86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:26.136748 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 552/1086] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:21:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:21:26.160560 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f910d6a-0c55-459d-86e6-51b10e1814c5 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:26.277379 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f910d6a-0c55-459d-86e6-51b10e1814c5 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:26.304019 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:26.312352 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950dfdaf-a757-402d-bb0b-3fa1e43dc072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:26.313162 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950dfdaf-a757-402d-bb0b-3fa1e43dc072 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:26.313510 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950dfdaf-a757-402d-bb0b-3fa1e43dc072 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:26.313790 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950dfdaf-a757-402d-bb0b-3fa1e43dc072 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:26.325207 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950dfdaf-a757-402d-bb0b-3fa1e43dc072 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:21:26.334161 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-950dfdaf-a757-402d-bb0b-3fa1e43dc072 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:21:26.334161 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:21:26.334161 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:21:26.334161 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:21:26.334161 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:21:26.334161 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:21:26.334161 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:21:26.334161 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:21:26.334161 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:21:26.334161 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:21:26.334161 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:21:26.334161 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:21:26.334161 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:21:26.334161 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:21:26.334161 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:21:26.334161 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:21:26.336612 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:21:26.336612 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:21:26.336612 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:21:26.336612 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:21:26.336612 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:21:26.336612 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:21:26.336612 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:21:26.336612 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:21:26.336612 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:21:26.336612 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:21:26.336612 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:21:26.336612 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:21:26.336612 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:21:26.336612 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:21:26.336612 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:21:26.336612 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:21:26.337470 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:21:26.337470 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:21:26.337470 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:21:26.337470 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:21:26.337470 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:21:26.337470 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:21:26.337470 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:21:26.337470 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:21:26.337470 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:21:26.337470 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:21:26.337470 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:21:26.337470 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:21:26.337470 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 535/1087] 10.222.0.85 () {56 vars in 1096 bytes} [Tue Jun 7 14:21:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:21:26.341359 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29748556-9674-4eec-85d8-d39700cdca86 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:26.393252 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99de62a-99b5-43af-b29e-40d0375a58c7 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:26.393519 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99de62a-99b5-43af-b29e-40d0375a58c7 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:26.393807 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99de62a-99b5-43af-b29e-40d0375a58c7 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:26.395172 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 536/1088] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:21:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:21:26.402001 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7785d76d-2879-4c75-be15-45b87634f8e7 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:26.402507 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7785d76d-2879-4c75-be15-45b87634f8e7 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:26.403006 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7785d76d-2879-4c75-be15-45b87634f8e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:26.428327 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-7785d76d-2879-4c75-be15-45b87634f8e7 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:26.430363 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-7785d76d-2879-4c75-be15-45b87634f8e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:26.441046 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:26.466413 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29748556-9674-4eec-85d8-d39700cdca86 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:26.502844 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366e2917-7321-467c-aaa2-9224e522503b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:26.503559 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e2917-7321-467c-aaa2-9224e522503b None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:26.503848 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e2917-7321-467c-aaa2-9224e522503b None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:26.504124 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e2917-7321-467c-aaa2-9224e522503b None admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:26.504898 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366e2917-7321-467c-aaa2-9224e522503b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:26.505233 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366e2917-7321-467c-aaa2-9224e522503b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:26.506603 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366e2917-7321-467c-aaa2-9224e522503b None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:26.514520 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 553/1089] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jun 7 14:21:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:21:26.519867 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 537/1090] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:21:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 14:21:26.520537 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366e2917-7321-467c-aaa2-9224e522503b None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:26.543387 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7785d76d-2879-4c75-be15-45b87634f8e7 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:26.622704 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366e2917-7321-467c-aaa2-9224e522503b None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:26.648656 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7785d76d-2879-4c75-be15-45b87634f8e7 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:26.657951 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5211bad3-2703-49ae-a38d-b709756f1feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:26.659174 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5211bad3-2703-49ae-a38d-b709756f1feb None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:26.659461 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5211bad3-2703-49ae-a38d-b709756f1feb None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:26.659737 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5211bad3-2703-49ae-a38d-b709756f1feb None admin] PATH_INFO: `/v3/users/58c51eb3059944b0b1dd8ebcf89650d0` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:26.660982 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5211bad3-2703-49ae-a38d-b709756f1feb None admin] RBAC: Authorizing `identity:get_user(user_id=58c51eb3059944b0b1dd8ebcf89650d0)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:26.663085 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5211bad3-2703-49ae-a38d-b709756f1feb None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:26.664785 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 554/1091] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jun 7 14:21:26 2022] GET /identity/v3/users/58c51eb3059944b0b1dd8ebcf89650d0 => generated 285 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:21:26.666928 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:26.670378 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5211bad3-2703-49ae-a38d-b709756f1feb None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:26.686536 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f848167-221e-4f0b-aa8b-8b03fa225891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:26.687806 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f848167-221e-4f0b-aa8b-8b03fa225891 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:26.688266 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f848167-221e-4f0b-aa8b-8b03fa225891 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:26.688752 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f848167-221e-4f0b-aa8b-8b03fa225891 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:26.698996 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:26.704287 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f848167-221e-4f0b-aa8b-8b03fa225891 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:21:26.719041 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-8f848167-221e-4f0b-aa8b-8b03fa225891 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 14:21:26.719041 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:21:26.719041 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:21:26.719041 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:21:26.719041 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:21:26.719041 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:21:26.719041 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:21:26.719041 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:21:26.719041 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:21:26.719041 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:21:26.719041 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:21:26.719041 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:21:26.719041 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:21:26.719041 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:21:26.719041 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:21:26.719041 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:21:26.720036 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:21:26.720036 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:21:26.720036 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:21:26.720036 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:21:26.720036 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:21:26.720036 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:21:26.720036 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:21:26.720036 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:21:26.720036 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:21:26.720036 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:21:26.720036 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:21:26.720036 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:21:26.720036 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:21:26.720036 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:21:26.720036 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:21:26.720036 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:21:26.720942 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:21:26.720942 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:21:26.720942 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:21:26.720942 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:21:26.720942 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:21:26.720942 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:21:26.720942 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:21:26.720942 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:21:26.720942 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:21:26.720942 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:21:26.720942 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 14:21:26.720942 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:21:26.721600 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 538/1092] 10.222.0.85 () {56 vars in 1120 bytes} [Tue Jun 7 14:21:26 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:21:26.727940 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f848167-221e-4f0b-aa8b-8b03fa225891 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:26.800387 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5211bad3-2703-49ae-a38d-b709756f1feb None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:26.830371 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f848167-221e-4f0b-aa8b-8b03fa225891 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:26.832656 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a3dc2d-92b1-4140-84b0-8e523ea395e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:26.833816 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a3dc2d-92b1-4140-84b0-8e523ea395e9 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:26.834100 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a3dc2d-92b1-4140-84b0-8e523ea395e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:26.834379 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a3dc2d-92b1-4140-84b0-8e523ea395e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:26.865431 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086b8eaa-ad14-4e4b-b243-a941fa47ad84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:26.866346 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086b8eaa-ad14-4e4b-b243-a941fa47ad84 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:26.866725 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086b8eaa-ad14-4e4b-b243-a941fa47ad84 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:26.867097 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086b8eaa-ad14-4e4b-b243-a941fa47ad84 None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:26.868020 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086b8eaa-ad14-4e4b-b243-a941fa47ad84 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:26.868742 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086b8eaa-ad14-4e4b-b243-a941fa47ad84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:26.870647 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086b8eaa-ad14-4e4b-b243-a941fa47ad84 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:26.878930 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 539/1093] 10.222.0.85 () {56 vars in 1115 bytes} [Tue Jun 7 14:21:26 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:21:26.885208 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086b8eaa-ad14-4e4b-b243-a941fa47ad84 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:26.902591 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a3dc2d-92b1-4140-84b0-8e523ea395e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:26.904032 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a3dc2d-92b1-4140-84b0-8e523ea395e9 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:26.916282 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:26.928410 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:26.997118 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086b8eaa-ad14-4e4b-b243-a941fa47ad84 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:27.014452 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 555/1094] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jun 7 14:21:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 07 14:21:27.020262 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a3dc2d-92b1-4140-84b0-8e523ea395e9 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:27.052498 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52524116-7dd5-491e-a91a-75a0bdb22a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:27.053787 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52524116-7dd5-491e-a91a-75a0bdb22a94 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:27.054302 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52524116-7dd5-491e-a91a-75a0bdb22a94 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:27.054654 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52524116-7dd5-491e-a91a-75a0bdb22a94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:27.065916 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52524116-7dd5-491e-a91a-75a0bdb22a94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:27.067373 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52524116-7dd5-491e-a91a-75a0bdb22a94 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:27.085597 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-52524116-7dd5-491e-a91a-75a0bdb22a94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:21:27.085597 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:21:27.085597 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:21:27.085597 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:21:27.085597 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:21:27.085597 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:21:27.085597 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:21:27.085597 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:21:27.085597 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:21:27.085597 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:21:27.085597 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:21:27.085597 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:21:27.085597 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:21:27.085597 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:21:27.085597 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:21:27.086601 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:21:27.086601 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:21:27.086601 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:21:27.086601 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:21:27.086601 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:21:27.086601 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:21:27.086601 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:21:27.086601 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:21:27.086601 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:21:27.086601 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:21:27.086601 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:21:27.086601 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:21:27.086601 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:21:27.086601 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:21:27.086601 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:21:27.086601 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:21:27.086601 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:21:27.087503 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:21:27.087503 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:21:27.087503 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:21:27.087503 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:21:27.087503 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 540/1095] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:21:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:21:27.093131 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52524116-7dd5-491e-a91a-75a0bdb22a94 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:27.122897 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a3dc2d-92b1-4140-84b0-8e523ea395e9 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:27.134968 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:27.162474 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e79aad-ffe8-4e61-b291-835a9e503ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:27.163706 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e79aad-ffe8-4e61-b291-835a9e503ba8 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:27.164036 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e79aad-ffe8-4e61-b291-835a9e503ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:27.164470 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e79aad-ffe8-4e61-b291-835a9e503ba8 None admin] PATH_INFO: `/v3/projects/8f2a8a9b8fce42f68f0ce7f1a365ae2e` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:27.166134 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e79aad-ffe8-4e61-b291-835a9e503ba8 None admin] RBAC: Authorizing `identity:get_project(project_id=8f2a8a9b8fce42f68f0ce7f1a365ae2e)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:27.168255 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e79aad-ffe8-4e61-b291-835a9e503ba8 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:27.171675 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 556/1096] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jun 7 14:21:27 2022] GET /identity/v3/projects/8f2a8a9b8fce42f68f0ce7f1a365ae2e => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:21:27.177097 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e79aad-ffe8-4e61-b291-835a9e503ba8 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:27.205919 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:27.216602 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52524116-7dd5-491e-a91a-75a0bdb22a94 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:27.254122 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5ccd6f-f4fd-4be0-b0fe-f16223625348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:27.255568 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5ccd6f-f4fd-4be0-b0fe-f16223625348 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:27.256067 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5ccd6f-f4fd-4be0-b0fe-f16223625348 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:27.256512 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5ccd6f-f4fd-4be0-b0fe-f16223625348 None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:27.257355 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5ccd6f-f4fd-4be0-b0fe-f16223625348 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:27.257906 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5ccd6f-f4fd-4be0-b0fe-f16223625348 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:27.259947 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5ccd6f-f4fd-4be0-b0fe-f16223625348 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:27.276643 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 541/1097] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jun 7 14:21:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:21:27.282632 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5ccd6f-f4fd-4be0-b0fe-f16223625348 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:27.287526 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e79aad-ffe8-4e61-b291-835a9e503ba8 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:27.330314 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2ee7eb-2733-4bc1-b1ec-ba37d478117b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:27.331172 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2ee7eb-2733-4bc1-b1ec-ba37d478117b None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:27.331602 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2ee7eb-2733-4bc1-b1ec-ba37d478117b None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:27.332069 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2ee7eb-2733-4bc1-b1ec-ba37d478117b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:27.334136 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2ee7eb-2733-4bc1-b1ec-ba37d478117b None admin] RBAC: Adding query filter params (role.id=9baff2b18f2649f7b757ff2e1f63c853, scope.project.id=8f2a8a9b8fce42f68f0ce7f1a365ae2e, user.id=58c51eb3059944b0b1dd8ebcf89650d0) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:27.334636 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2ee7eb-2733-4bc1-b1ec-ba37d478117b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:27.337165 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2ee7eb-2733-4bc1-b1ec-ba37d478117b None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:27.346782 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 557/1098] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jun 7 14:21:27 2022] GET /identity/v3/role_assignments?user.id=58c51eb3059944b0b1dd8ebcf89650d0&scope.project.id=8f2a8a9b8fce42f68f0ce7f1a365ae2e&role.id=9baff2b18f2649f7b757ff2e1f63c853 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:21:27.413891 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5ccd6f-f4fd-4be0-b0fe-f16223625348 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:27.454112 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcb0bff-a315-42f1-85ad-6a143647b955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:27.454926 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcb0bff-a315-42f1-85ad-6a143647b955 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:27.455212 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcb0bff-a315-42f1-85ad-6a143647b955 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:27.455489 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcb0bff-a315-42f1-85ad-6a143647b955 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:27.464416 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcb0bff-a315-42f1-85ad-6a143647b955 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:27.465904 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcb0bff-a315-42f1-85ad-6a143647b955 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:27.483871 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-1fcb0bff-a315-42f1-85ad-6a143647b955 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 07 14:21:27.483871 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:21:27.483871 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:21:27.483871 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:21:27.483871 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:21:27.483871 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:21:27.483871 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:21:27.483871 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:21:27.483871 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:21:27.483871 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:21:27.483871 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:21:27.483871 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:21:27.483871 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 14:21:27.483871 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 14:21:27.483871 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 14:21:27.483871 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 14:21:27.485147 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:21:27.485147 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:21:27.485147 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 14:21:27.485147 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:21:27.485147 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 14:21:27.485147 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:21:27.485147 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:21:27.485147 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:21:27.485147 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:21:27.485147 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:21:27.485147 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:21:27.485147 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:21:27.485147 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:21:27.485147 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:21:27.485147 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:21:27.485147 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:21:27.485147 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:21:27.485147 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:21:27.486156 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:21:27.486156 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:21:27.486156 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 14:21:27.486156 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 14:21:27.486156 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 14:21:27.486156 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 14:21:27.486156 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 14:21:27.486156 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 14:21:27.486156 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 07 14:21:27.486156 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:21:27.486156 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 542/1099] 10.222.0.85 () {56 vars in 1093 bytes} [Tue Jun 7 14:21:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:21:27.494154 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2ee7eb-2733-4bc1-b1ec-ba37d478117b None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:27.560134 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:27.631689 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2ee7eb-2733-4bc1-b1ec-ba37d478117b None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:27.680272 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dc662d-09b8-4213-80d0-9b21caac6683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:27.681146 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dc662d-09b8-4213-80d0-9b21caac6683 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:27.681541 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dc662d-09b8-4213-80d0-9b21caac6683 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:27.681983 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dc662d-09b8-4213-80d0-9b21caac6683 None admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:27.682836 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dc662d-09b8-4213-80d0-9b21caac6683 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:27.683153 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dc662d-09b8-4213-80d0-9b21caac6683 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:27.684603 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dc662d-09b8-4213-80d0-9b21caac6683 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:27.706973 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 558/1100] 10.222.0.85 () {56 vars in 1133 bytes} [Tue Jun 7 14:21:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:21:27.713603 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcb0bff-a315-42f1-85ad-6a143647b955 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:27.727354 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:27.832089 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcb0bff-a315-42f1-85ad-6a143647b955 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:27.878991 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312b8cce-6217-490c-b192-6800effffa4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:27.879941 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312b8cce-6217-490c-b192-6800effffa4a None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:27.881018 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312b8cce-6217-490c-b192-6800effffa4a None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:27.881439 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312b8cce-6217-490c-b192-6800effffa4a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:27.956366 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312b8cce-6217-490c-b192-6800effffa4a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:27.957957 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312b8cce-6217-490c-b192-6800effffa4a None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:27.970962 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:28.080099 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 543/1101] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jun 7 14:21:27 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 14:21:28.086036 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dc662d-09b8-4213-80d0-9b21caac6683 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:28.190937 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:28.194894 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dc662d-09b8-4213-80d0-9b21caac6683 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:28.232664 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078fd0fd-59e9-4cd8-88e9-541bb5470b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:28.233899 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078fd0fd-59e9-4cd8-88e9-541bb5470b12 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:28.234396 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078fd0fd-59e9-4cd8-88e9-541bb5470b12 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:28.234782 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078fd0fd-59e9-4cd8-88e9-541bb5470b12 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:28.246614 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078fd0fd-59e9-4cd8-88e9-541bb5470b12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:28.248083 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078fd0fd-59e9-4cd8-88e9-541bb5470b12 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:28.266207 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-078fd0fd-59e9-4cd8-88e9-541bb5470b12 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:21:28.266207 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:21:28.266207 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:21:28.266207 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:21:28.266207 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:21:28.266207 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:21:28.266207 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:21:28.266207 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:21:28.266207 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:21:28.266207 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:21:28.266207 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:21:28.266207 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:21:28.266207 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:21:28.266207 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:21:28.266207 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:21:28.267276 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:21:28.267276 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:21:28.267276 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:21:28.267276 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:21:28.267276 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:21:28.267276 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:21:28.267276 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:21:28.267276 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:21:28.267276 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:21:28.267276 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:21:28.267276 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:21:28.267276 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:21:28.267276 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:21:28.267276 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:21:28.267276 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:21:28.267276 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:21:28.267276 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:21:28.268854 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:21:28.268854 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:21:28.268854 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:21:28.268854 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:21:28.268854 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 559/1102] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:21:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:21:28.273868 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312b8cce-6217-490c-b192-6800effffa4a None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:28.375613 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312b8cce-6217-490c-b192-6800effffa4a None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:28.387521 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:28.412130 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c160d992-a8ec-4f60-a50f-bbcf38ff6fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:28.412818 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c160d992-a8ec-4f60-a50f-bbcf38ff6fac None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:28.413099 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c160d992-a8ec-4f60-a50f-bbcf38ff6fac None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:28.413376 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c160d992-a8ec-4f60-a50f-bbcf38ff6fac None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:28.414127 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c160d992-a8ec-4f60-a50f-bbcf38ff6fac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:28.414428 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c160d992-a8ec-4f60-a50f-bbcf38ff6fac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:28.415879 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c160d992-a8ec-4f60-a50f-bbcf38ff6fac None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:28.428133 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 544/1103] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jun 7 14:21:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:21:28.434518 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078fd0fd-59e9-4cd8-88e9-541bb5470b12 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:28.547120 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078fd0fd-59e9-4cd8-88e9-541bb5470b12 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:28.577287 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb21c78-f923-4cdb-a812-dfd1b1015e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:28.578096 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb21c78-f923-4cdb-a812-dfd1b1015e8f None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:28.578390 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb21c78-f923-4cdb-a812-dfd1b1015e8f None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:28.578668 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb21c78-f923-4cdb-a812-dfd1b1015e8f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:28.586571 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb21c78-f923-4cdb-a812-dfd1b1015e8f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:28.588017 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb21c78-f923-4cdb-a812-dfd1b1015e8f None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:28.602301 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-1fb21c78-f923-4cdb-a812-dfd1b1015e8f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:21:28.602301 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:21:28.602301 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:21:28.602301 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:21:28.602301 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:21:28.602301 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:21:28.602301 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:21:28.602301 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:21:28.602301 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:21:28.602301 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:21:28.602301 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:21:28.602301 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:21:28.602301 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 14:21:28.602301 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 14:21:28.602301 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 14:21:28.603430 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 14:21:28.603430 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:21:28.603430 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:21:28.603430 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:21:28.603430 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:21:28.603430 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:21:28.603430 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:21:28.603430 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:21:28.603430 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:21:28.603430 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:21:28.603430 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:21:28.603430 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:21:28.603430 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:21:28.603430 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:21:28.603430 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:21:28.603430 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:21:28.603430 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:21:28.604713 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 14:21:28.604713 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 14:21:28.604713 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 14:21:28.604713 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 14:21:28.604713 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 14:21:28.604713 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 14:21:28.604713 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:21:28.604713 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:21:28.608245 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 560/1104] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jun 7 14:21:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:21:28.609222 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c160d992-a8ec-4f60-a50f-bbcf38ff6fac None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:28.714759 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c160d992-a8ec-4f60-a50f-bbcf38ff6fac None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:28.750246 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6651a6b-78e2-4579-a480-b1b81036268c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:28.750920 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6651a6b-78e2-4579-a480-b1b81036268c None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:28.751207 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6651a6b-78e2-4579-a480-b1b81036268c None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:28.751485 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6651a6b-78e2-4579-a480-b1b81036268c None admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:28.752281 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6651a6b-78e2-4579-a480-b1b81036268c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:28.752622 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6651a6b-78e2-4579-a480-b1b81036268c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:28.754112 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6651a6b-78e2-4579-a480-b1b81036268c None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:28.770088 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 545/1105] 10.222.0.85 () {56 vars in 1148 bytes} [Tue Jun 7 14:21:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:21:28.777212 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb21c78-f923-4cdb-a812-dfd1b1015e8f None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:28.792968 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:28.885289 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb21c78-f923-4cdb-a812-dfd1b1015e8f None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:28.919071 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278606d7-1566-4d65-b9c3-bdc076fed0d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:28.919715 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278606d7-1566-4d65-b9c3-bdc076fed0d8 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:28.920001 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278606d7-1566-4d65-b9c3-bdc076fed0d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:28.920282 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278606d7-1566-4d65-b9c3-bdc076fed0d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:28.921141 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278606d7-1566-4d65-b9c3-bdc076fed0d8 None admin] RBAC: Adding query filter params (role.id=5ca16aa9be5740d2902e27a98242edf4, scope.project.id=724785eb74e44daf8736fd01fc0d1839, user.id=8e857db4902e42989d0d040bbc8517c8) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:28.921453 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278606d7-1566-4d65-b9c3-bdc076fed0d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:28.922883 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278606d7-1566-4d65-b9c3-bdc076fed0d8 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:28.929301 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 561/1106] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jun 7 14:21:28 2022] GET /identity/v3/role_assignments?user.id=8e857db4902e42989d0d040bbc8517c8&scope.project.id=724785eb74e44daf8736fd01fc0d1839&role.id=5ca16aa9be5740d2902e27a98242edf4 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:21:29.730587 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ea98d4-d103-4e01-bb2a-0e532654c90d None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:29.731331 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ea98d4-d103-4e01-bb2a-0e532654c90d None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:29.731331 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ea98d4-d103-4e01-bb2a-0e532654c90d None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:29.732846 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 546/1107] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:21:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:21:29.738506 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2718f024-eef9-4586-bdf7-6e0f8aea4c2b None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:29.738958 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2718f024-eef9-4586-bdf7-6e0f8aea4c2b None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:29.739083 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2718f024-eef9-4586-bdf7-6e0f8aea4c2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:29.760046 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-2718f024-eef9-4586-bdf7-6e0f8aea4c2b None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:29.761747 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-2718f024-eef9-4586-bdf7-6e0f8aea4c2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:29.845473 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 562/1108] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:21:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 14:21:29.868054 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ea98d4-d103-4e01-bb2a-0e532654c90d None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:29.906601 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:30.001710 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ea98d4-d103-4e01-bb2a-0e532654c90d None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:30.037160 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fc6395-64b3-4cd7-9d78-f849b3e51453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:30.037968 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fc6395-64b3-4cd7-9d78-f849b3e51453 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:30.038251 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fc6395-64b3-4cd7-9d78-f849b3e51453 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:30.038527 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fc6395-64b3-4cd7-9d78-f849b3e51453 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:30.039700 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fc6395-64b3-4cd7-9d78-f849b3e51453 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:30.041162 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fc6395-64b3-4cd7-9d78-f849b3e51453 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:30.042756 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 547/1109] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:21:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:21:30.048515 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2718f024-eef9-4586-bdf7-6e0f8aea4c2b None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:30.058215 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:30.155921 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2718f024-eef9-4586-bdf7-6e0f8aea4c2b None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:30.189664 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d725858c-e4f7-4ebe-829d-8226267903c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:30.190336 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d725858c-e4f7-4ebe-829d-8226267903c6 None admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:30.190625 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d725858c-e4f7-4ebe-829d-8226267903c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:30.190906 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d725858c-e4f7-4ebe-829d-8226267903c6 None admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:30.191614 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d725858c-e4f7-4ebe-829d-8226267903c6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:30.193635 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d725858c-e4f7-4ebe-829d-8226267903c6 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:30.344524 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 563/1110] 10.222.0.85 () {60 vars in 1132 bytes} [Tue Jun 7 14:21:30 2022] POST /identity/v3/users => generated 286 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:21:32.628201 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0d1810-2009-4d62-8c6b-9b4f6aa5cabf None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:32.628804 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0d1810-2009-4d62-8c6b-9b4f6aa5cabf None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:32.628804 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0d1810-2009-4d62-8c6b-9b4f6aa5cabf None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:32.630166 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 548/1111] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:21:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:21:32.635429 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e00b7d-9168-44ba-9107-61641a23f989 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:32.635863 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e00b7d-9168-44ba-9107-61641a23f989 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:32.635987 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e00b7d-9168-44ba-9107-61641a23f989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:32.652655 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-94e00b7d-9168-44ba-9107-61641a23f989 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:32.653828 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-94e00b7d-9168-44ba-9107-61641a23f989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:32.709876 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:32.720241 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 564/1112] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:21:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:21:32.744726 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0d1810-2009-4d62-8c6b-9b4f6aa5cabf None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:32.861311 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:32.867788 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0d1810-2009-4d62-8c6b-9b4f6aa5cabf None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:32.909119 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f214ea0-5662-4126-b0c0-c41d23fa5e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:32.909887 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f214ea0-5662-4126-b0c0-c41d23fa5e4c None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:32.910175 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f214ea0-5662-4126-b0c0-c41d23fa5e4c None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:32.910441 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f214ea0-5662-4126-b0c0-c41d23fa5e4c None admin] PATH_INFO: `/v3/roles/494cd000d6884a179c334848f52a3a21` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:32.912739 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f214ea0-5662-4126-b0c0-c41d23fa5e4c None admin] RBAC: Authorizing `identity:get_role(role_id=494cd000d6884a179c334848f52a3a21)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:32.914639 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f214ea0-5662-4126-b0c0-c41d23fa5e4c None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:32.916000 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 549/1113] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jun 7 14:21:32 2022] GET /identity/v3/roles/494cd000d6884a179c334848f52a3a21 => generated 223 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:21:32.921958 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e00b7d-9168-44ba-9107-61641a23f989 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:33.020264 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e00b7d-9168-44ba-9107-61641a23f989 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:33.054370 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382f419e-c7fd-47c4-be25-a1986219159e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:33.055235 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382f419e-c7fd-47c4-be25-a1986219159e None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:33.055520 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382f419e-c7fd-47c4-be25-a1986219159e None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:33.055813 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382f419e-c7fd-47c4-be25-a1986219159e None admin] PATH_INFO: `/v3/users/3c8f09f2e19b4bda9b81cd9bbef38d25` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:33.069031 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382f419e-c7fd-47c4-be25-a1986219159e None admin] RBAC: Authorizing `identity:get_user(user_id=3c8f09f2e19b4bda9b81cd9bbef38d25)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:33.070465 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382f419e-c7fd-47c4-be25-a1986219159e None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:33.072173 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 565/1114] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jun 7 14:21:32 2022] GET /identity/v3/users/3c8f09f2e19b4bda9b81cd9bbef38d25 => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:21:33.079484 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f214ea0-5662-4126-b0c0-c41d23fa5e4c None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:33.188266 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f214ea0-5662-4126-b0c0-c41d23fa5e4c None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:33.227895 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c06a01-0b34-47c5-a41f-cbea2994f40b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:33.228737 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c06a01-0b34-47c5-a41f-cbea2994f40b None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:33.229016 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c06a01-0b34-47c5-a41f-cbea2994f40b None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:33.229501 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c06a01-0b34-47c5-a41f-cbea2994f40b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:33.269917 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:33.312563 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c06a01-0b34-47c5-a41f-cbea2994f40b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:33.314115 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c06a01-0b34-47c5-a41f-cbea2994f40b None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:33.427934 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 550/1115] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jun 7 14:21:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 14:21:33.434080 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382f419e-c7fd-47c4-be25-a1986219159e None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:33.500280 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:33.550955 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382f419e-c7fd-47c4-be25-a1986219159e None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:33.593028 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c170bd3e-e9b6-4948-8be8-a8ac31a4539f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:33.593904 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c170bd3e-e9b6-4948-8be8-a8ac31a4539f None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:33.594188 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c170bd3e-e9b6-4948-8be8-a8ac31a4539f None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:33.594493 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c170bd3e-e9b6-4948-8be8-a8ac31a4539f None admin] PATH_INFO: `/v3/projects/8f2a8a9b8fce42f68f0ce7f1a365ae2e` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:33.595936 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c170bd3e-e9b6-4948-8be8-a8ac31a4539f None admin] RBAC: Authorizing `identity:get_project(project_id=8f2a8a9b8fce42f68f0ce7f1a365ae2e)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:33.597522 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c170bd3e-e9b6-4948-8be8-a8ac31a4539f None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:33.600851 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 566/1116] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jun 7 14:21:33 2022] GET /identity/v3/projects/8f2a8a9b8fce42f68f0ce7f1a365ae2e => generated 311 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:21:33.605766 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c06a01-0b34-47c5-a41f-cbea2994f40b None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:33.678809 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:33.714621 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c06a01-0b34-47c5-a41f-cbea2994f40b None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:33.750162 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67476c7e-2940-4521-af13-f706b0edd372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:33.750946 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67476c7e-2940-4521-af13-f706b0edd372 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:33.751215 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67476c7e-2940-4521-af13-f706b0edd372 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:33.751548 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67476c7e-2940-4521-af13-f706b0edd372 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:33.752481 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67476c7e-2940-4521-af13-f706b0edd372 None admin] RBAC: Adding query filter params (role.id=494cd000d6884a179c334848f52a3a21, scope.project.id=8f2a8a9b8fce42f68f0ce7f1a365ae2e, user.id=3c8f09f2e19b4bda9b81cd9bbef38d25) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:33.752848 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67476c7e-2940-4521-af13-f706b0edd372 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:33.754325 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67476c7e-2940-4521-af13-f706b0edd372 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:33.760528 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 551/1117] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jun 7 14:21:33 2022] GET /identity/v3/role_assignments?user.id=3c8f09f2e19b4bda9b81cd9bbef38d25&scope.project.id=8f2a8a9b8fce42f68f0ce7f1a365ae2e&role.id=494cd000d6884a179c334848f52a3a21 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:21:36.001178 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c219e3dc-671f-4770-86fb-bc4a7096d7e8 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:36.001731 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c219e3dc-671f-4770-86fb-bc4a7096d7e8 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:36.001963 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c219e3dc-671f-4770-86fb-bc4a7096d7e8 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:36.003587 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 567/1118] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:21:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:21:36.009300 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c6d981-1d6b-467c-a7a0-2c75051e8fba None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:36.009710 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c6d981-1d6b-467c-a7a0-2c75051e8fba None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:36.009855 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c6d981-1d6b-467c-a7a0-2c75051e8fba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:36.026592 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-33c6d981-1d6b-467c-a7a0-2c75051e8fba None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:36.027774 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-33c6d981-1d6b-467c-a7a0-2c75051e8fba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:36.108188 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 552/1119] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:21:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:21:36.132521 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c219e3dc-671f-4770-86fb-bc4a7096d7e8 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:36.250903 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c219e3dc-671f-4770-86fb-bc4a7096d7e8 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:36.269135 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:36.286280 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7ee349-3339-4259-a8e9-0ddaa22bec56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:36.287244 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7ee349-3339-4259-a8e9-0ddaa22bec56 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:36.287531 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7ee349-3339-4259-a8e9-0ddaa22bec56 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:36.287809 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7ee349-3339-4259-a8e9-0ddaa22bec56 None admin] PATH_INFO: `/v3/roles/494cd000d6884a179c334848f52a3a21` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:36.289332 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7ee349-3339-4259-a8e9-0ddaa22bec56 None admin] RBAC: Authorizing `identity:get_role(role_id=494cd000d6884a179c334848f52a3a21)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:36.290726 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7ee349-3339-4259-a8e9-0ddaa22bec56 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:36.292179 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 568/1120] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jun 7 14:21:36 2022] GET /identity/v3/roles/494cd000d6884a179c334848f52a3a21 => generated 223 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:21:36.298281 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c6d981-1d6b-467c-a7a0-2c75051e8fba None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:36.360563 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:36.406545 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c6d981-1d6b-467c-a7a0-2c75051e8fba None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:36.443828 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d979bfb0-1716-456f-b265-e7ab75662779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:36.444724 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d979bfb0-1716-456f-b265-e7ab75662779 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:36.445014 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d979bfb0-1716-456f-b265-e7ab75662779 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:36.445297 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d979bfb0-1716-456f-b265-e7ab75662779 None admin] PATH_INFO: `/v3/users/3c8f09f2e19b4bda9b81cd9bbef38d25` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:36.446500 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d979bfb0-1716-456f-b265-e7ab75662779 None admin] RBAC: Authorizing `identity:get_user(user_id=3c8f09f2e19b4bda9b81cd9bbef38d25)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:36.447927 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d979bfb0-1716-456f-b265-e7ab75662779 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:36.449598 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 553/1121] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jun 7 14:21:36 2022] GET /identity/v3/users/3c8f09f2e19b4bda9b81cd9bbef38d25 => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:21:36.455529 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7ee349-3339-4259-a8e9-0ddaa22bec56 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:36.559570 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7ee349-3339-4259-a8e9-0ddaa22bec56 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:36.593510 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beea336b-e3a8-49ee-83bd-8e2bed235aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:36.594372 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beea336b-e3a8-49ee-83bd-8e2bed235aba None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:36.594655 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beea336b-e3a8-49ee-83bd-8e2bed235aba None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:36.594932 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beea336b-e3a8-49ee-83bd-8e2bed235aba None admin] PATH_INFO: `/v3/projects/8f2a8a9b8fce42f68f0ce7f1a365ae2e` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:36.596113 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beea336b-e3a8-49ee-83bd-8e2bed235aba None admin] RBAC: Authorizing `identity:get_project(project_id=8f2a8a9b8fce42f68f0ce7f1a365ae2e)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:36.597579 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beea336b-e3a8-49ee-83bd-8e2bed235aba None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:36.599302 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 569/1122] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jun 7 14:21:36 2022] GET /identity/v3/projects/8f2a8a9b8fce42f68f0ce7f1a365ae2e => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:21:36.605286 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d979bfb0-1716-456f-b265-e7ab75662779 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:36.712431 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d979bfb0-1716-456f-b265-e7ab75662779 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:36.738549 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:36.747024 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a08be1d-c12d-4902-a982-fe8d5e24fd6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:36.747910 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a08be1d-c12d-4902-a982-fe8d5e24fd6a None admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:36.748194 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a08be1d-c12d-4902-a982-fe8d5e24fd6a None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:36.753907 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a08be1d-c12d-4902-a982-fe8d5e24fd6a None admin] PATH_INFO: `/v3/projects/8f2a8a9b8fce42f68f0ce7f1a365ae2e/users/3c8f09f2e19b4bda9b81cd9bbef38d25/roles/494cd000d6884a179c334848f52a3a21` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:36.753907 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a08be1d-c12d-4902-a982-fe8d5e24fd6a None admin] RBAC: Authorizing `identity:create_grant(project_id=8f2a8a9b8fce42f68f0ce7f1a365ae2e, user_id=3c8f09f2e19b4bda9b81cd9bbef38d25, role_id=494cd000d6884a179c334848f52a3a21)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:36.753907 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a08be1d-c12d-4902-a982-fe8d5e24fd6a None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:36.830581 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 554/1123] 10.222.0.85 () {58 vars in 1439 bytes} [Tue Jun 7 14:21:36 2022] PUT /identity/v3/projects/8f2a8a9b8fce42f68f0ce7f1a365ae2e/users/3c8f09f2e19b4bda9b81cd9bbef38d25/roles/494cd000d6884a179c334848f52a3a21 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:21:39.103293 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eeb8e0-d48d-481b-a822-7b400aaa1e34 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:39.103828 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eeb8e0-d48d-481b-a822-7b400aaa1e34 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:39.103920 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eeb8e0-d48d-481b-a822-7b400aaa1e34 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:39.105287 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 570/1124] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:21:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:21:39.111188 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbfcfb6-d48b-43b7-a992-9d3a2163954e None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:39.111623 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbfcfb6-d48b-43b7-a992-9d3a2163954e None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:39.111897 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbfcfb6-d48b-43b7-a992-9d3a2163954e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:39.139633 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-7bbfcfb6-d48b-43b7-a992-9d3a2163954e None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:39.141642 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-7bbfcfb6-d48b-43b7-a992-9d3a2163954e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:39.247388 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 555/1125] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:21:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:21:39.281511 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eeb8e0-d48d-481b-a822-7b400aaa1e34 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:39.342162 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:39.397691 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eeb8e0-d48d-481b-a822-7b400aaa1e34 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:39.437753 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab011904-f7b1-4bc7-986c-7b629cc1272a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:39.438622 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab011904-f7b1-4bc7-986c-7b629cc1272a None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:39.438907 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab011904-f7b1-4bc7-986c-7b629cc1272a None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:39.439189 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab011904-f7b1-4bc7-986c-7b629cc1272a None admin] PATH_INFO: `/v3/roles/494cd000d6884a179c334848f52a3a21` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:39.440734 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab011904-f7b1-4bc7-986c-7b629cc1272a None admin] RBAC: Authorizing `identity:get_role(role_id=494cd000d6884a179c334848f52a3a21)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:39.442355 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab011904-f7b1-4bc7-986c-7b629cc1272a None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:39.443761 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 571/1126] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jun 7 14:21:39 2022] GET /identity/v3/roles/494cd000d6884a179c334848f52a3a21 => generated 223 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:21:39.449772 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbfcfb6-d48b-43b7-a992-9d3a2163954e None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:39.558530 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbfcfb6-d48b-43b7-a992-9d3a2163954e None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:39.576423 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:39.594256 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50163eae-5d20-470f-a5ce-f705f9436258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:39.595137 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50163eae-5d20-470f-a5ce-f705f9436258 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:39.595414 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50163eae-5d20-470f-a5ce-f705f9436258 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:39.595757 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50163eae-5d20-470f-a5ce-f705f9436258 None admin] PATH_INFO: `/v3/users/3c8f09f2e19b4bda9b81cd9bbef38d25` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:39.596996 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50163eae-5d20-470f-a5ce-f705f9436258 None admin] RBAC: Authorizing `identity:get_user(user_id=3c8f09f2e19b4bda9b81cd9bbef38d25)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:39.598472 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50163eae-5d20-470f-a5ce-f705f9436258 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:39.600694 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 556/1127] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jun 7 14:21:39 2022] GET /identity/v3/users/3c8f09f2e19b4bda9b81cd9bbef38d25 => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:21:39.606507 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab011904-f7b1-4bc7-986c-7b629cc1272a None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:39.727382 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab011904-f7b1-4bc7-986c-7b629cc1272a None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:39.750120 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:39.761653 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f2f02b-f266-4852-9941-3897f1d8b470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:39.762303 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f2f02b-f266-4852-9941-3897f1d8b470 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:39.762590 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f2f02b-f266-4852-9941-3897f1d8b470 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:39.762867 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f2f02b-f266-4852-9941-3897f1d8b470 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:39.837293 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f2f02b-f266-4852-9941-3897f1d8b470 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:39.838654 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f2f02b-f266-4852-9941-3897f1d8b470 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:39.940920 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 572/1128] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jun 7 14:21:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 14:21:39.947840 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50163eae-5d20-470f-a5ce-f705f9436258 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:40.056643 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50163eae-5d20-470f-a5ce-f705f9436258 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:40.092630 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4a5d7d-6c3a-40d3-86cb-9897f93ad0d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:40.093409 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4a5d7d-6c3a-40d3-86cb-9897f93ad0d6 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:40.093693 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4a5d7d-6c3a-40d3-86cb-9897f93ad0d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:40.093975 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4a5d7d-6c3a-40d3-86cb-9897f93ad0d6 None admin] PATH_INFO: `/v3/projects/8f2a8a9b8fce42f68f0ce7f1a365ae2e` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:40.095191 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4a5d7d-6c3a-40d3-86cb-9897f93ad0d6 None admin] RBAC: Authorizing `identity:get_project(project_id=8f2a8a9b8fce42f68f0ce7f1a365ae2e)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:40.096644 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4a5d7d-6c3a-40d3-86cb-9897f93ad0d6 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:40.098371 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 557/1129] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jun 7 14:21:39 2022] GET /identity/v3/projects/8f2a8a9b8fce42f68f0ce7f1a365ae2e => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:21:40.104679 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f2f02b-f266-4852-9941-3897f1d8b470 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:40.129034 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:40.208697 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f2f02b-f266-4852-9941-3897f1d8b470 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:40.242816 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8138b2e-8dd5-49bf-99e3-5d769f8a71ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:40.243514 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8138b2e-8dd5-49bf-99e3-5d769f8a71ca None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:40.243797 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8138b2e-8dd5-49bf-99e3-5d769f8a71ca None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:40.244072 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8138b2e-8dd5-49bf-99e3-5d769f8a71ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:40.244956 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8138b2e-8dd5-49bf-99e3-5d769f8a71ca None admin] RBAC: Adding query filter params (role.id=494cd000d6884a179c334848f52a3a21, scope.project.id=8f2a8a9b8fce42f68f0ce7f1a365ae2e, user.id=3c8f09f2e19b4bda9b81cd9bbef38d25) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:40.245523 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8138b2e-8dd5-49bf-99e3-5d769f8a71ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:40.246953 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8138b2e-8dd5-49bf-99e3-5d769f8a71ca None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:40.252935 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 573/1130] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jun 7 14:21:40 2022] GET /identity/v3/role_assignments?user.id=3c8f09f2e19b4bda9b81cd9bbef38d25&scope.project.id=8f2a8a9b8fce42f68f0ce7f1a365ae2e&role.id=494cd000d6884a179c334848f52a3a21 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:21:42.583972 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd73221d-f28d-4a9d-b027-c9fcdfc27a48 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:42.584696 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd73221d-f28d-4a9d-b027-c9fcdfc27a48 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:42.584696 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd73221d-f28d-4a9d-b027-c9fcdfc27a48 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:42.585932 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 558/1131] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:21:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 14:21:42.592053 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d6619-0c6f-476c-9f57-c5c003bfd7f5 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:42.592530 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d6619-0c6f-476c-9f57-c5c003bfd7f5 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:42.592915 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d6619-0c6f-476c-9f57-c5c003bfd7f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:42.611295 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-db8d6619-0c6f-476c-9f57-c5c003bfd7f5 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:42.612547 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-db8d6619-0c6f-476c-9f57-c5c003bfd7f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:42.809564 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 574/1132] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:21:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 14:21:42.833900 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd73221d-f28d-4a9d-b027-c9fcdfc27a48 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:42.895207 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:42.943587 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd73221d-f28d-4a9d-b027-c9fcdfc27a48 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:42.984971 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57022468-8ab8-486c-b574-6fe0aa5d7e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:42.985757 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57022468-8ab8-486c-b574-6fe0aa5d7e17 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:42.986050 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57022468-8ab8-486c-b574-6fe0aa5d7e17 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:42.986324 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57022468-8ab8-486c-b574-6fe0aa5d7e17 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:42.987535 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57022468-8ab8-486c-b574-6fe0aa5d7e17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:42.989021 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57022468-8ab8-486c-b574-6fe0aa5d7e17 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:42.990630 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 559/1133] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:21:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:21:42.996716 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8d6619-0c6f-476c-9f57-c5c003bfd7f5 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:43.014569 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:43.094874 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8d6619-0c6f-476c-9f57-c5c003bfd7f5 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:43.127913 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6657b5e-22a9-4493-a7ac-b5fab5badafb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:43.128598 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6657b5e-22a9-4493-a7ac-b5fab5badafb None admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:43.128882 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6657b5e-22a9-4493-a7ac-b5fab5badafb None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:43.129157 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6657b5e-22a9-4493-a7ac-b5fab5badafb None admin] PATH_INFO: `/v3/projects` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:43.130376 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6657b5e-22a9-4493-a7ac-b5fab5badafb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:43.131792 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6657b5e-22a9-4493-a7ac-b5fab5badafb None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:43.249405 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 575/1134] 10.222.0.85 () {60 vars in 1141 bytes} [Tue Jun 7 14:21:42 2022] POST /identity/v3/projects => generated 311 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:21:45.559300 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b0153d-ba05-40e5-8674-3932a945bb15 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:45.559867 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b0153d-ba05-40e5-8674-3932a945bb15 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:45.559867 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b0153d-ba05-40e5-8674-3932a945bb15 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:45.561235 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 560/1135] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:21:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:21:45.566667 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ae407-9001-4595-8f22-fe4b9609e55c None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:45.567635 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ae407-9001-4595-8f22-fe4b9609e55c None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:45.567839 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ae407-9001-4595-8f22-fe4b9609e55c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:45.584204 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-f83ae407-9001-4595-8f22-fe4b9609e55c None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:45.585395 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-f83ae407-9001-4595-8f22-fe4b9609e55c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:45.645135 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:45.650680 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 576/1136] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:21:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:21:45.675164 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b0153d-ba05-40e5-8674-3932a945bb15 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:45.775897 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b0153d-ba05-40e5-8674-3932a945bb15 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:45.797340 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:45.808777 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07967d4b-445f-48e1-9186-b328a3ce916a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:45.809608 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07967d4b-445f-48e1-9186-b328a3ce916a None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:45.809838 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07967d4b-445f-48e1-9186-b328a3ce916a None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:45.810114 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07967d4b-445f-48e1-9186-b328a3ce916a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:45.811284 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07967d4b-445f-48e1-9186-b328a3ce916a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:45.812720 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07967d4b-445f-48e1-9186-b328a3ce916a None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:45.814302 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 561/1137] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:21:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:21:45.820696 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83ae407-9001-4595-8f22-fe4b9609e55c None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:45.916770 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83ae407-9001-4595-8f22-fe4b9609e55c None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:45.950084 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cd646d-3f51-4fbc-ac17-a4179aae362e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:45.950753 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cd646d-3f51-4fbc-ac17-a4179aae362e None admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:45.951041 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cd646d-3f51-4fbc-ac17-a4179aae362e None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:45.951322 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cd646d-3f51-4fbc-ac17-a4179aae362e None admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:45.952030 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cd646d-3f51-4fbc-ac17-a4179aae362e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:45.953491 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cd646d-3f51-4fbc-ac17-a4179aae362e None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:46.041110 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 577/1138] 10.222.0.85 () {60 vars in 1132 bytes} [Tue Jun 7 14:21:45 2022] POST /identity/v3/users => generated 286 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:21:48.321417 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cebf24-f7a0-4c59-af44-c112f2a56574 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:48.321933 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cebf24-f7a0-4c59-af44-c112f2a56574 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:48.321933 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cebf24-f7a0-4c59-af44-c112f2a56574 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:48.323254 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 562/1139] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:21:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:21:48.328946 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703336fa-96d6-4678-9f68-c56bf6cd12cb None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:48.329516 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703336fa-96d6-4678-9f68-c56bf6cd12cb None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:48.329516 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703336fa-96d6-4678-9f68-c56bf6cd12cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:48.352470 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-703336fa-96d6-4678-9f68-c56bf6cd12cb None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:48.353418 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-703336fa-96d6-4678-9f68-c56bf6cd12cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:48.407882 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:48.431173 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 578/1140] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:21:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 14:21:48.454567 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cebf24-f7a0-4c59-af44-c112f2a56574 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:48.555651 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cebf24-f7a0-4c59-af44-c112f2a56574 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:48.589588 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0dc8ac-53b1-4dad-a155-c1b063ed5204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:48.590287 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0dc8ac-53b1-4dad-a155-c1b063ed5204 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:48.590571 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0dc8ac-53b1-4dad-a155-c1b063ed5204 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:48.590847 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0dc8ac-53b1-4dad-a155-c1b063ed5204 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:48.600585 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0dc8ac-53b1-4dad-a155-c1b063ed5204 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:21:48.607991 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-bd0dc8ac-53b1-4dad-a155-c1b063ed5204 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:21:48.607991 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:21:48.607991 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:21:48.607991 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:21:48.607991 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:21:48.607991 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:21:48.607991 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:21:48.607991 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:21:48.607991 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:21:48.607991 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:21:48.607991 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:21:48.607991 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:21:48.607991 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:21:48.607991 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:21:48.607991 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:21:48.607991 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:21:48.609113 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:21:48.609113 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:21:48.609113 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:21:48.609113 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:21:48.609113 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:21:48.609113 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:21:48.609113 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:21:48.609113 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:21:48.609113 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:21:48.609113 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:21:48.609113 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:21:48.609113 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:21:48.609113 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:21:48.609113 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:21:48.609113 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:21:48.609113 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:21:48.609987 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:21:48.609987 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:21:48.609987 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:21:48.609987 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:21:48.609987 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:21:48.609987 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:21:48.609987 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:21:48.609987 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:21:48.609987 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:21:48.609987 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:21:48.609987 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:21:48.609987 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:21:48.609987 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 563/1141] 10.222.0.85 () {56 vars in 1096 bytes} [Tue Jun 7 14:21:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:21:48.615701 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703336fa-96d6-4678-9f68-c56bf6cd12cb None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:48.714600 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703336fa-96d6-4678-9f68-c56bf6cd12cb None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:48.749131 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0e357b-0467-4512-8d8b-ec33c5cfcb99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:48.749801 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0e357b-0467-4512-8d8b-ec33c5cfcb99 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:48.750086 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0e357b-0467-4512-8d8b-ec33c5cfcb99 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:48.750363 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0e357b-0467-4512-8d8b-ec33c5cfcb99 None admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:48.751116 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0e357b-0467-4512-8d8b-ec33c5cfcb99 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:48.751423 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0e357b-0467-4512-8d8b-ec33c5cfcb99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:48.752930 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0e357b-0467-4512-8d8b-ec33c5cfcb99 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:48.761644 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 579/1142] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jun 7 14:21:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:21:48.767424 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0dc8ac-53b1-4dad-a155-c1b063ed5204 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:48.817715 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:48.866862 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0dc8ac-53b1-4dad-a155-c1b063ed5204 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:48.899585 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afca1779-b9a9-40ac-b7bc-53e470f1664a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:48.900535 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afca1779-b9a9-40ac-b7bc-53e470f1664a None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:48.900821 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afca1779-b9a9-40ac-b7bc-53e470f1664a None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:48.901093 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afca1779-b9a9-40ac-b7bc-53e470f1664a None admin] PATH_INFO: `/v3/users/84467483a49546c881b150297f539d23` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:48.914215 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afca1779-b9a9-40ac-b7bc-53e470f1664a None admin] RBAC: Authorizing `identity:get_user(user_id=84467483a49546c881b150297f539d23)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:48.915652 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afca1779-b9a9-40ac-b7bc-53e470f1664a None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:48.917303 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 564/1143] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jun 7 14:21:48 2022] GET /identity/v3/users/84467483a49546c881b150297f539d23 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:21:48.923055 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0e357b-0467-4512-8d8b-ec33c5cfcb99 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:48.968359 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:49.032877 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0e357b-0467-4512-8d8b-ec33c5cfcb99 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:49.067703 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bd67f3-b5b2-44b1-b318-04a538e46cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:49.068351 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd67f3-b5b2-44b1-b318-04a538e46cb1 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:49.068657 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd67f3-b5b2-44b1-b318-04a538e46cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:49.068934 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd67f3-b5b2-44b1-b318-04a538e46cb1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:49.135229 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bd67f3-b5b2-44b1-b318-04a538e46cb1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:49.136631 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bd67f3-b5b2-44b1-b318-04a538e46cb1 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:49.179630 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:49.230583 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 580/1144] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jun 7 14:21:48 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 14:21:49.236584 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afca1779-b9a9-40ac-b7bc-53e470f1664a None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:49.335050 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afca1779-b9a9-40ac-b7bc-53e470f1664a None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:49.345986 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:49.370617 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523e88b8-357e-4ae7-9841-466fc8fe6ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:49.371463 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523e88b8-357e-4ae7-9841-466fc8fe6ecb None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:49.371745 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523e88b8-357e-4ae7-9841-466fc8fe6ecb None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:49.372020 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523e88b8-357e-4ae7-9841-466fc8fe6ecb None admin] PATH_INFO: `/v3/projects/92474fd595db40048ab19d1bb51b1adc` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:49.373221 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523e88b8-357e-4ae7-9841-466fc8fe6ecb None admin] RBAC: Authorizing `identity:get_project(project_id=92474fd595db40048ab19d1bb51b1adc)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:49.374628 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523e88b8-357e-4ae7-9841-466fc8fe6ecb None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:49.376287 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 565/1145] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jun 7 14:21:49 2022] GET /identity/v3/projects/92474fd595db40048ab19d1bb51b1adc => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:21:49.382405 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bd67f3-b5b2-44b1-b318-04a538e46cb1 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:49.476704 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bd67f3-b5b2-44b1-b318-04a538e46cb1 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:49.512092 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901600b0-0cf7-4d88-9e5b-416012ab00f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:49.512759 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901600b0-0cf7-4d88-9e5b-416012ab00f5 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:49.513038 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901600b0-0cf7-4d88-9e5b-416012ab00f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:49.513310 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901600b0-0cf7-4d88-9e5b-416012ab00f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:49.514133 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901600b0-0cf7-4d88-9e5b-416012ab00f5 None admin] RBAC: Adding query filter params (role.id=9baff2b18f2649f7b757ff2e1f63c853, scope.project.id=92474fd595db40048ab19d1bb51b1adc, user.id=84467483a49546c881b150297f539d23) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:49.514441 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901600b0-0cf7-4d88-9e5b-416012ab00f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:49.515855 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901600b0-0cf7-4d88-9e5b-416012ab00f5 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:49.521676 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 581/1146] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jun 7 14:21:49 2022] GET /identity/v3/role_assignments?user.id=84467483a49546c881b150297f539d23&scope.project.id=92474fd595db40048ab19d1bb51b1adc&role.id=9baff2b18f2649f7b757ff2e1f63c853 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:21:51.744811 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9676098-963d-437d-ae5e-3ad65f7bc968 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:51.745626 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9676098-963d-437d-ae5e-3ad65f7bc968 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:51.745626 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9676098-963d-437d-ae5e-3ad65f7bc968 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:51.746957 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 566/1147] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:21:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:21:51.752240 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b61176-ef7a-40dd-907d-64b3420d01d2 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:51.752662 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b61176-ef7a-40dd-907d-64b3420d01d2 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:51.752840 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b61176-ef7a-40dd-907d-64b3420d01d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:51.769814 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-d5b61176-ef7a-40dd-907d-64b3420d01d2 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:51.771028 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-d5b61176-ef7a-40dd-907d-64b3420d01d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:51.773780 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:51.837123 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 582/1148] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:21:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:21:51.859744 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9676098-963d-437d-ae5e-3ad65f7bc968 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:51.959840 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9676098-963d-437d-ae5e-3ad65f7bc968 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:51.995426 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b2b74b-849f-4b3c-a1d5-579c340b0db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:51.996151 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b2b74b-849f-4b3c-a1d5-579c340b0db4 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:51.996472 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b2b74b-849f-4b3c-a1d5-579c340b0db4 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:51.996754 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b2b74b-849f-4b3c-a1d5-579c340b0db4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:52.006860 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b2b74b-849f-4b3c-a1d5-579c340b0db4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:21:52.014521 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-a0b2b74b-849f-4b3c-a1d5-579c340b0db4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:21:52.014521 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:21:52.014521 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:21:52.014521 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:21:52.014521 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:21:52.014521 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:21:52.014521 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:21:52.014521 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:21:52.014521 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:21:52.014521 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:21:52.014521 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:21:52.014521 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:21:52.014521 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:21:52.014521 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:21:52.014521 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:21:52.014521 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:21:52.015689 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:21:52.015689 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:21:52.015689 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:21:52.015689 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:21:52.015689 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:21:52.015689 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:21:52.015689 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:21:52.015689 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:21:52.015689 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:21:52.015689 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:21:52.015689 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:21:52.015689 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:21:52.015689 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:21:52.015689 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:21:52.015689 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:21:52.015689 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:21:52.016644 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:21:52.016644 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:21:52.016644 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:21:52.016644 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:21:52.016644 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:21:52.016644 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:21:52.016644 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:21:52.016644 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:21:52.016644 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:21:52.016644 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:21:52.016644 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:21:52.016644 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:21:52.016644 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 567/1149] 10.222.0.85 () {56 vars in 1096 bytes} [Tue Jun 7 14:21:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:21:52.021726 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b61176-ef7a-40dd-907d-64b3420d01d2 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:52.118931 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b61176-ef7a-40dd-907d-64b3420d01d2 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:52.152281 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329dffde-ee21-4aa1-b03b-678b14275f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:52.152968 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329dffde-ee21-4aa1-b03b-678b14275f17 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:52.153253 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329dffde-ee21-4aa1-b03b-678b14275f17 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:52.153532 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329dffde-ee21-4aa1-b03b-678b14275f17 None admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:52.154283 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329dffde-ee21-4aa1-b03b-678b14275f17 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:52.154593 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329dffde-ee21-4aa1-b03b-678b14275f17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:52.155958 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329dffde-ee21-4aa1-b03b-678b14275f17 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:52.164537 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 583/1150] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jun 7 14:21:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:21:52.170593 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b2b74b-849f-4b3c-a1d5-579c340b0db4 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:52.205736 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:52.268848 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b2b74b-849f-4b3c-a1d5-579c340b0db4 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:52.302018 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36389d84-90db-4bb9-bfb7-192226fb80c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:52.302887 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36389d84-90db-4bb9-bfb7-192226fb80c2 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:52.303175 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36389d84-90db-4bb9-bfb7-192226fb80c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:52.303456 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36389d84-90db-4bb9-bfb7-192226fb80c2 None admin] PATH_INFO: `/v3/users/84467483a49546c881b150297f539d23` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:52.304669 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36389d84-90db-4bb9-bfb7-192226fb80c2 None admin] RBAC: Authorizing `identity:get_user(user_id=84467483a49546c881b150297f539d23)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:52.306096 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36389d84-90db-4bb9-bfb7-192226fb80c2 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:52.307733 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 568/1151] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jun 7 14:21:52 2022] GET /identity/v3/users/84467483a49546c881b150297f539d23 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:21:52.313783 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329dffde-ee21-4aa1-b03b-678b14275f17 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:52.316702 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:52.413512 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329dffde-ee21-4aa1-b03b-678b14275f17 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:52.444912 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d05797-994a-4c94-9f0b-89d6f0c74657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:52.445747 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d05797-994a-4c94-9f0b-89d6f0c74657 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:52.446027 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d05797-994a-4c94-9f0b-89d6f0c74657 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:52.446299 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d05797-994a-4c94-9f0b-89d6f0c74657 None admin] PATH_INFO: `/v3/projects/92474fd595db40048ab19d1bb51b1adc` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:52.447454 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d05797-994a-4c94-9f0b-89d6f0c74657 None admin] RBAC: Authorizing `identity:get_project(project_id=92474fd595db40048ab19d1bb51b1adc)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:52.448905 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d05797-994a-4c94-9f0b-89d6f0c74657 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:52.450573 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 584/1152] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jun 7 14:21:52 2022] GET /identity/v3/projects/92474fd595db40048ab19d1bb51b1adc => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:21:52.456486 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36389d84-90db-4bb9-bfb7-192226fb80c2 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:52.555547 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36389d84-90db-4bb9-bfb7-192226fb80c2 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:52.557766 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:52.587899 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be5f1ee-eaee-45eb-9acc-64a26197b0ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:52.588830 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be5f1ee-eaee-45eb-9acc-64a26197b0ed None admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:52.589324 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be5f1ee-eaee-45eb-9acc-64a26197b0ed None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:52.589782 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be5f1ee-eaee-45eb-9acc-64a26197b0ed None admin] PATH_INFO: `/v3/projects/92474fd595db40048ab19d1bb51b1adc/users/84467483a49546c881b150297f539d23/roles/9baff2b18f2649f7b757ff2e1f63c853` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:52.592006 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be5f1ee-eaee-45eb-9acc-64a26197b0ed None admin] RBAC: Authorizing `identity:create_grant(project_id=92474fd595db40048ab19d1bb51b1adc, user_id=84467483a49546c881b150297f539d23, role_id=9baff2b18f2649f7b757ff2e1f63c853)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:52.593474 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be5f1ee-eaee-45eb-9acc-64a26197b0ed None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:52.692757 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 569/1153] 10.222.0.85 () {58 vars in 1439 bytes} [Tue Jun 7 14:21:52 2022] PUT /identity/v3/projects/92474fd595db40048ab19d1bb51b1adc/users/84467483a49546c881b150297f539d23/roles/9baff2b18f2649f7b757ff2e1f63c853 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:21:54.918771 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9afa2b-b956-4308-8a72-da49005e194b None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:54.919305 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9afa2b-b956-4308-8a72-da49005e194b None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:54.919415 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9afa2b-b956-4308-8a72-da49005e194b None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:54.920750 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 585/1154] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:21:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:21:54.925923 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000927e8-2a57-4971-93be-f7f888bc6aaf None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:54.926882 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000927e8-2a57-4971-93be-f7f888bc6aaf None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:54.927058 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000927e8-2a57-4971-93be-f7f888bc6aaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:54.943789 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-000927e8-2a57-4971-93be-f7f888bc6aaf None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:54.945004 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-000927e8-2a57-4971-93be-f7f888bc6aaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:55.011332 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 570/1155] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:21:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:21:55.034593 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9afa2b-b956-4308-8a72-da49005e194b None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:55.117813 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:55.148757 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9afa2b-b956-4308-8a72-da49005e194b None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:55.182148 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151e942e-0769-479b-aa6b-696e956506c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:55.182869 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151e942e-0769-479b-aa6b-696e956506c0 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:55.183154 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151e942e-0769-479b-aa6b-696e956506c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:55.183434 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151e942e-0769-479b-aa6b-696e956506c0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:55.193375 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151e942e-0769-479b-aa6b-696e956506c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:21:55.200932 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-151e942e-0769-479b-aa6b-696e956506c0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:21:55.200932 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:21:55.200932 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:21:55.200932 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:21:55.200932 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:21:55.200932 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:21:55.200932 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:21:55.200932 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:21:55.200932 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:21:55.200932 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:21:55.200932 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:21:55.200932 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:21:55.200932 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:21:55.200932 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:21:55.200932 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:21:55.200932 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:21:55.202046 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:21:55.202046 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:21:55.202046 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:21:55.202046 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:21:55.202046 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:21:55.202046 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:21:55.202046 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:21:55.202046 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:21:55.202046 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:21:55.202046 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:21:55.202046 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:21:55.202046 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:21:55.202046 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:21:55.202046 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:21:55.202046 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:21:55.202046 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:21:55.203046 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:21:55.203046 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:21:55.203046 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:21:55.203046 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:21:55.203046 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:21:55.203046 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:21:55.203046 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:21:55.203046 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:21:55.203046 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:21:55.203046 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:21:55.203046 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:21:55.203046 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:21:55.203046 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 586/1156] 10.222.0.85 () {56 vars in 1096 bytes} [Tue Jun 7 14:21:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:21:55.208175 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000927e8-2a57-4971-93be-f7f888bc6aaf None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:55.299557 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:55.306105 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000927e8-2a57-4971-93be-f7f888bc6aaf None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:55.340610 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa74cbd-314e-47e0-9a15-79c8e1f62a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:55.341278 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa74cbd-314e-47e0-9a15-79c8e1f62a29 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:55.341568 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa74cbd-314e-47e0-9a15-79c8e1f62a29 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:55.341930 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa74cbd-314e-47e0-9a15-79c8e1f62a29 None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:55.342679 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa74cbd-314e-47e0-9a15-79c8e1f62a29 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:55.342994 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa74cbd-314e-47e0-9a15-79c8e1f62a29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:55.344351 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa74cbd-314e-47e0-9a15-79c8e1f62a29 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:55.353042 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 571/1157] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jun 7 14:21:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:21:55.359154 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151e942e-0769-479b-aa6b-696e956506c0 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:55.460955 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151e942e-0769-479b-aa6b-696e956506c0 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:55.488421 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:55.494443 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7663107-0054-4b91-8788-410e81fa4105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:55.495618 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7663107-0054-4b91-8788-410e81fa4105 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:55.495902 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7663107-0054-4b91-8788-410e81fa4105 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:55.496176 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7663107-0054-4b91-8788-410e81fa4105 None admin] PATH_INFO: `/v3/users/84467483a49546c881b150297f539d23` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:55.497414 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7663107-0054-4b91-8788-410e81fa4105 None admin] RBAC: Authorizing `identity:get_user(user_id=84467483a49546c881b150297f539d23)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:55.499359 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7663107-0054-4b91-8788-410e81fa4105 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:55.500521 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 587/1158] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jun 7 14:21:55 2022] GET /identity/v3/users/84467483a49546c881b150297f539d23 => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:21:55.506271 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa74cbd-314e-47e0-9a15-79c8e1f62a29 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:55.605547 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa74cbd-314e-47e0-9a15-79c8e1f62a29 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:55.638547 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d987b3-fb9f-434f-a552-9b635c5a645b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:55.639190 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d987b3-fb9f-434f-a552-9b635c5a645b None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:55.639471 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d987b3-fb9f-434f-a552-9b635c5a645b None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:55.639748 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d987b3-fb9f-434f-a552-9b635c5a645b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:55.664321 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:55.706031 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d987b3-fb9f-434f-a552-9b635c5a645b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:55.708029 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d987b3-fb9f-434f-a552-9b635c5a645b None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:55.804288 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 572/1159] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jun 7 14:21:55 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 14:21:55.810156 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7663107-0054-4b91-8788-410e81fa4105 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:55.907303 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7663107-0054-4b91-8788-410e81fa4105 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:55.940610 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d05b34-23ab-43ed-a9b2-da3faf3c39a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:55.941550 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d05b34-23ab-43ed-a9b2-da3faf3c39a1 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:55.942356 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d05b34-23ab-43ed-a9b2-da3faf3c39a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:55.942632 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d05b34-23ab-43ed-a9b2-da3faf3c39a1 None admin] PATH_INFO: `/v3/projects/92474fd595db40048ab19d1bb51b1adc` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:55.943819 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d05b34-23ab-43ed-a9b2-da3faf3c39a1 None admin] RBAC: Authorizing `identity:get_project(project_id=92474fd595db40048ab19d1bb51b1adc)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:55.945295 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d05b34-23ab-43ed-a9b2-da3faf3c39a1 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:55.946997 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 588/1160] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jun 7 14:21:55 2022] GET /identity/v3/projects/92474fd595db40048ab19d1bb51b1adc => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:21:55.953112 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d987b3-fb9f-434f-a552-9b635c5a645b None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:56.016009 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:56.057865 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d987b3-fb9f-434f-a552-9b635c5a645b None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:56.091837 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6e14f8-23b2-459b-a15a-7e47642963be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:56.092514 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6e14f8-23b2-459b-a15a-7e47642963be None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:56.092802 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6e14f8-23b2-459b-a15a-7e47642963be None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:56.093083 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6e14f8-23b2-459b-a15a-7e47642963be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:56.093921 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6e14f8-23b2-459b-a15a-7e47642963be None admin] RBAC: Adding query filter params (role.id=9baff2b18f2649f7b757ff2e1f63c853, scope.project.id=92474fd595db40048ab19d1bb51b1adc, user.id=84467483a49546c881b150297f539d23) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:56.094235 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6e14f8-23b2-459b-a15a-7e47642963be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:56.095655 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6e14f8-23b2-459b-a15a-7e47642963be None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:56.102039 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 573/1161] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jun 7 14:21:55 2022] GET /identity/v3/role_assignments?user.id=84467483a49546c881b150297f539d23&scope.project.id=92474fd595db40048ab19d1bb51b1adc&role.id=9baff2b18f2649f7b757ff2e1f63c853 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:21:58.383694 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae301fc8-d011-4ebd-86fd-70c42ddc27b6 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:58.384218 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae301fc8-d011-4ebd-86fd-70c42ddc27b6 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:58.384218 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae301fc8-d011-4ebd-86fd-70c42ddc27b6 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:58.385579 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 589/1162] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:21:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:21:58.391110 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98153524-62a7-41c1-8d79-910bfdd896ca None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:58.391573 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98153524-62a7-41c1-8d79-910bfdd896ca None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:58.391660 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98153524-62a7-41c1-8d79-910bfdd896ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:58.408281 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-98153524-62a7-41c1-8d79-910bfdd896ca None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:21:58.409497 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-98153524-62a7-41c1-8d79-910bfdd896ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:21:58.474813 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 574/1163] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:21:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:21:58.497862 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae301fc8-d011-4ebd-86fd-70c42ddc27b6 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:58.565142 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:58.601171 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae301fc8-d011-4ebd-86fd-70c42ddc27b6 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:58.635172 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e202d370-91f3-457c-a12c-12ee8096dab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:58.635742 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e202d370-91f3-457c-a12c-12ee8096dab5 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:58.636031 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e202d370-91f3-457c-a12c-12ee8096dab5 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:58.636309 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e202d370-91f3-457c-a12c-12ee8096dab5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:58.702360 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e202d370-91f3-457c-a12c-12ee8096dab5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:58.703795 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e202d370-91f3-457c-a12c-12ee8096dab5 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:58.766563 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:58.800691 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 590/1164] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jun 7 14:21:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 14:21:58.807280 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98153524-62a7-41c1-8d79-910bfdd896ca None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:58.857120 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:21:58.906083 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98153524-62a7-41c1-8d79-910bfdd896ca None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:58.940340 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11c8d9a-96fc-42eb-9d09-3e3853d6bc8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:58.941176 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11c8d9a-96fc-42eb-9d09-3e3853d6bc8d None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:58.941457 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11c8d9a-96fc-42eb-9d09-3e3853d6bc8d None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:58.941773 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11c8d9a-96fc-42eb-9d09-3e3853d6bc8d None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:58.948550 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11c8d9a-96fc-42eb-9d09-3e3853d6bc8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:58.949969 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11c8d9a-96fc-42eb-9d09-3e3853d6bc8d None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:58.963686 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-e11c8d9a-96fc-42eb-9d09-3e3853d6bc8d None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 07 14:21:58.963686 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:21:58.963686 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 07 14:21:58.963686 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 07 14:21:58.963686 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 14:21:58.963686 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 14:21:58.963686 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 14:21:58.963686 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 14:21:58.963686 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 07 14:21:58.963686 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:21:58.963686 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 14:21:58.963686 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:21:58.963686 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:21:58.963686 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:21:58.963686 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:21:58.963686 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:21:58.963686 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:21:58.964928 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:21:58.964928 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:21:58.964928 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:21:58.964928 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:21:58.964928 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:21:58.964928 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:21:58.964928 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:21:58.964928 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:21:58.964928 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:21:58.964928 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:21:58.964928 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:21:58.964928 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:21:58.964928 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:21:58.964928 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:21:58.964928 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:21:58.964928 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:21:58.965801 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:21:58.965801 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:21:58.965801 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:21:58.965801 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:21:58.965801 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:21:58.965801 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:21:58.965801 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:21:58.965801 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:21:58.965801 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:21:58.965801 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:21:58.965801 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 07 14:21:58.965801 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:21:58.965801 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 07 14:21:58.965801 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:21:58.965801 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 575/1165] 10.222.0.85 () {56 vars in 1117 bytes} [Tue Jun 7 14:21:58 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:21:58.970843 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e202d370-91f3-457c-a12c-12ee8096dab5 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:21:59.067477 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e202d370-91f3-457c-a12c-12ee8096dab5 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:21:59.100846 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef86f7a-d6e3-45df-a84e-ba3928c64285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:21:59.101501 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef86f7a-d6e3-45df-a84e-ba3928c64285 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:21:59.101782 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef86f7a-d6e3-45df-a84e-ba3928c64285 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:21:59.102061 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef86f7a-d6e3-45df-a84e-ba3928c64285 None admin] PATH_INFO: `/v3/domains` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:21:59.102776 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef86f7a-d6e3-45df-a84e-ba3928c64285 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:21:59.103076 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef86f7a-d6e3-45df-a84e-ba3928c64285 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:21:59.104441 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef86f7a-d6e3-45df-a84e-ba3928c64285 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:21:59.111651 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 591/1166] 10.222.0.85 () {56 vars in 1115 bytes} [Tue Jun 7 14:21:58 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:22:01.357266 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffe415f-a8c4-4cb1-bd21-55792207df95 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:01.357772 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffe415f-a8c4-4cb1-bd21-55792207df95 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:01.357772 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffe415f-a8c4-4cb1-bd21-55792207df95 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:01.359085 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 576/1167] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:22:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:22:01.365077 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee5b233-806e-4595-b240-f447b77d8231 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:01.365490 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee5b233-806e-4595-b240-f447b77d8231 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:01.365644 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee5b233-806e-4595-b240-f447b77d8231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:01.382931 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-bee5b233-806e-4595-b240-f447b77d8231 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:22:01.384108 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-bee5b233-806e-4595-b240-f447b77d8231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:01.406660 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:01.449597 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 592/1168] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:22:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:22:01.472543 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffe415f-a8c4-4cb1-bd21-55792207df95 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:01.571935 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffe415f-a8c4-4cb1-bd21-55792207df95 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:01.581965 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:01.604430 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b428766f-4f28-4eb9-8b86-3061117dcad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:01.605172 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b428766f-4f28-4eb9-8b86-3061117dcad1 None admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:01.605459 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b428766f-4f28-4eb9-8b86-3061117dcad1 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:01.605739 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b428766f-4f28-4eb9-8b86-3061117dcad1 None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:01.606481 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b428766f-4f28-4eb9-8b86-3061117dcad1 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:01.607887 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b428766f-4f28-4eb9-8b86-3061117dcad1 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:01.681877 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 577/1169] 10.222.0.85 () {60 vars in 1138 bytes} [Tue Jun 7 14:22:01 2022] POST /identity/v3/domains => generated 267 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:22:03.949191 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a2b378-67e9-4bbb-8313-7797f112d651 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:03.949729 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a2b378-67e9-4bbb-8313-7797f112d651 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:03.949729 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a2b378-67e9-4bbb-8313-7797f112d651 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:03.951130 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 593/1170] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:22:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:22:03.956523 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8446f606-64ad-40f7-ba66-521b403c5d8b None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:03.956980 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8446f606-64ad-40f7-ba66-521b403c5d8b None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:03.957072 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8446f606-64ad-40f7-ba66-521b403c5d8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:03.975529 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-8446f606-64ad-40f7-ba66-521b403c5d8b None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:22:03.976785 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-8446f606-64ad-40f7-ba66-521b403c5d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:04.050108 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 578/1171] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:22:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:22:04.075252 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a2b378-67e9-4bbb-8313-7797f112d651 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:04.190089 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a2b378-67e9-4bbb-8313-7797f112d651 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:04.224628 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747e3077-b78c-4d25-9fff-8331a84cd1c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:04.225500 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747e3077-b78c-4d25-9fff-8331a84cd1c6 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:04.225790 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747e3077-b78c-4d25-9fff-8331a84cd1c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:04.226075 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747e3077-b78c-4d25-9fff-8331a84cd1c6 None admin] PATH_INFO: `/v3/domains/153079d2b8a2404a9a5e009440492478` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:04.236614 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747e3077-b78c-4d25-9fff-8331a84cd1c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=153079d2b8a2404a9a5e009440492478)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:04.238071 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747e3077-b78c-4d25-9fff-8331a84cd1c6 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:04.240362 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 594/1172] 10.222.0.85 () {56 vars in 1183 bytes} [Tue Jun 7 14:22:04 2022] GET /identity/v3/domains/153079d2b8a2404a9a5e009440492478 => generated 267 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:22:04.246735 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8446f606-64ad-40f7-ba66-521b403c5d8b None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:04.353423 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:04.356605 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8446f606-64ad-40f7-ba66-521b403c5d8b None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:04.396349 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602cd0dc-7e5e-4c62-acdd-816260283426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:04.397035 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602cd0dc-7e5e-4c62-acdd-816260283426 None admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:04.397317 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602cd0dc-7e5e-4c62-acdd-816260283426 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:04.397595 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602cd0dc-7e5e-4c62-acdd-816260283426 None admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:04.398347 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602cd0dc-7e5e-4c62-acdd-816260283426 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:04.399736 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602cd0dc-7e5e-4c62-acdd-816260283426 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:04.528625 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 579/1173] 10.222.0.85 () {60 vars in 1141 bytes} [Tue Jun 7 14:22:04 2022] POST /identity/v3/projects => generated 361 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:22:06.849168 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7c7ae3-6f77-47e3-b805-a444d3b1d2f3 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:06.849744 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7c7ae3-6f77-47e3-b805-a444d3b1d2f3 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:06.849744 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7c7ae3-6f77-47e3-b805-a444d3b1d2f3 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:06.851201 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 595/1174] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:22:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:22:06.856659 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605e9151-eabd-48a4-a47a-0ede0b32dd0d None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:06.857090 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605e9151-eabd-48a4-a47a-0ede0b32dd0d None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:06.857090 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605e9151-eabd-48a4-a47a-0ede0b32dd0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:06.874436 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-605e9151-eabd-48a4-a47a-0ede0b32dd0d None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:22:06.875623 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-605e9151-eabd-48a4-a47a-0ede0b32dd0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:06.942782 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 580/1175] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:22:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:22:06.967664 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7c7ae3-6f77-47e3-b805-a444d3b1d2f3 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:06.991201 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:07.069662 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7c7ae3-6f77-47e3-b805-a444d3b1d2f3 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:07.103667 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d6be5f-5568-4708-b321-200992fd562c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:07.104485 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d6be5f-5568-4708-b321-200992fd562c None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:07.104776 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d6be5f-5568-4708-b321-200992fd562c None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:07.105058 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d6be5f-5568-4708-b321-200992fd562c None admin] PATH_INFO: `/v3/domains/153079d2b8a2404a9a5e009440492478` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:07.106246 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d6be5f-5568-4708-b321-200992fd562c None admin] RBAC: Authorizing `identity:get_domain(domain_id=153079d2b8a2404a9a5e009440492478)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:07.107681 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d6be5f-5568-4708-b321-200992fd562c None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:07.109361 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 596/1176] 10.222.0.85 () {56 vars in 1183 bytes} [Tue Jun 7 14:22:06 2022] GET /identity/v3/domains/153079d2b8a2404a9a5e009440492478 => generated 267 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:22:07.115687 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605e9151-eabd-48a4-a47a-0ede0b32dd0d None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:07.201053 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:07.219760 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605e9151-eabd-48a4-a47a-0ede0b32dd0d None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:07.254938 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc163f6-df22-4f53-8b6a-c775867ef5af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:07.255603 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc163f6-df22-4f53-8b6a-c775867ef5af None admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:07.255893 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc163f6-df22-4f53-8b6a-c775867ef5af None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:07.256169 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc163f6-df22-4f53-8b6a-c775867ef5af None admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:07.256931 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc163f6-df22-4f53-8b6a-c775867ef5af None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:07.258357 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc163f6-df22-4f53-8b6a-c775867ef5af None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:07.413307 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 581/1177] 10.222.0.85 () {60 vars in 1132 bytes} [Tue Jun 7 14:22:07 2022] POST /identity/v3/users => generated 311 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:22:09.749793 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5619aecf-4432-4d7b-a9d7-8e32b22318b1 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:09.750405 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5619aecf-4432-4d7b-a9d7-8e32b22318b1 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:09.750520 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5619aecf-4432-4d7b-a9d7-8e32b22318b1 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:09.752090 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 597/1178] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:22:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:22:09.758173 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bdc19f-a7e1-4f61-8fcb-9117f3e5ae0b None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:09.758633 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bdc19f-a7e1-4f61-8fcb-9117f3e5ae0b None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:09.758724 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bdc19f-a7e1-4f61-8fcb-9117f3e5ae0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:09.776592 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-13bdc19f-a7e1-4f61-8fcb-9117f3e5ae0b None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:22:09.777794 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-13bdc19f-a7e1-4f61-8fcb-9117f3e5ae0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:09.846907 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 582/1179] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:22:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:22:09.871239 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5619aecf-4432-4d7b-a9d7-8e32b22318b1 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:09.961777 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:09.974530 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5619aecf-4432-4d7b-a9d7-8e32b22318b1 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:10.008960 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a616975-b4f9-4626-926a-45bd28cb4d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:10.009643 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a616975-b4f9-4626-926a-45bd28cb4d17 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:10.009923 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a616975-b4f9-4626-926a-45bd28cb4d17 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:10.010188 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a616975-b4f9-4626-926a-45bd28cb4d17 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:10.019990 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a616975-b4f9-4626-926a-45bd28cb4d17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:22:10.027391 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-4a616975-b4f9-4626-926a-45bd28cb4d17 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:22:10.027391 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:22:10.027391 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:22:10.027391 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:22:10.027391 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:22:10.027391 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:22:10.027391 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:22:10.027391 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:22:10.027391 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:22:10.027391 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:22:10.027391 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:22:10.027391 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:22:10.027391 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:22:10.027391 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:22:10.027391 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:22:10.027391 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:22:10.028552 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:22:10.028552 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:22:10.028552 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:22:10.028552 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:22:10.028552 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:22:10.028552 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:22:10.028552 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:22:10.028552 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:22:10.028552 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:22:10.028552 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:22:10.028552 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:22:10.028552 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:22:10.028552 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:22:10.028552 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:22:10.028552 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:22:10.028552 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:22:10.034248 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:22:10.034248 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:22:10.034248 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:22:10.034248 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:22:10.034248 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:22:10.034248 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:22:10.034248 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:22:10.034248 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:22:10.034248 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:22:10.034248 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:22:10.034248 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:22:10.034248 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:22:10.034248 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 598/1180] 10.222.0.85 () {56 vars in 1096 bytes} [Tue Jun 7 14:22:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:22:10.035620 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bdc19f-a7e1-4f61-8fcb-9117f3e5ae0b None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:10.100127 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:10.133209 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bdc19f-a7e1-4f61-8fcb-9117f3e5ae0b None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:10.167415 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8bf214-74f7-4dd9-aaa7-796a9b039f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:10.168089 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8bf214-74f7-4dd9-aaa7-796a9b039f74 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:10.168491 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8bf214-74f7-4dd9-aaa7-796a9b039f74 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:10.168780 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8bf214-74f7-4dd9-aaa7-796a9b039f74 None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:10.169528 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8bf214-74f7-4dd9-aaa7-796a9b039f74 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:22:10.169837 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8bf214-74f7-4dd9-aaa7-796a9b039f74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:10.171330 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8bf214-74f7-4dd9-aaa7-796a9b039f74 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:10.183755 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 583/1181] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jun 7 14:22:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:22:10.190208 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a616975-b4f9-4626-926a-45bd28cb4d17 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:10.372198 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a616975-b4f9-4626-926a-45bd28cb4d17 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:10.404931 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f605f3a-44f4-46f0-b0a0-8feeabc9a6f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:10.405782 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f605f3a-44f4-46f0-b0a0-8feeabc9a6f2 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:10.406068 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f605f3a-44f4-46f0-b0a0-8feeabc9a6f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:10.406348 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f605f3a-44f4-46f0-b0a0-8feeabc9a6f2 None admin] PATH_INFO: `/v3/users/49414326a00b4a32a2d8aa68f68c9a83` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:10.420881 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f605f3a-44f4-46f0-b0a0-8feeabc9a6f2 None admin] RBAC: Authorizing `identity:get_user(user_id=49414326a00b4a32a2d8aa68f68c9a83)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:10.422330 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f605f3a-44f4-46f0-b0a0-8feeabc9a6f2 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:10.424010 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 599/1182] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jun 7 14:22:10 2022] GET /identity/v3/users/49414326a00b4a32a2d8aa68f68c9a83 => generated 311 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:22:10.429914 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8bf214-74f7-4dd9-aaa7-796a9b039f74 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:10.527272 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8bf214-74f7-4dd9-aaa7-796a9b039f74 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:10.563502 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:10.569592 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd8b65e-d8df-4def-b3bd-e179e9653345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:10.570245 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd8b65e-d8df-4def-b3bd-e179e9653345 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:10.570531 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd8b65e-d8df-4def-b3bd-e179e9653345 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:10.570802 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd8b65e-d8df-4def-b3bd-e179e9653345 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:10.637628 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd8b65e-d8df-4def-b3bd-e179e9653345 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:10.638983 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd8b65e-d8df-4def-b3bd-e179e9653345 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:10.737579 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 584/1183] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jun 7 14:22:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 14:22:10.744073 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f605f3a-44f4-46f0-b0a0-8feeabc9a6f2 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:10.753188 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:10.857124 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f605f3a-44f4-46f0-b0a0-8feeabc9a6f2 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:10.891376 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f42e405-f89f-41f4-9350-8b7a27874cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:10.892152 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f42e405-f89f-41f4-9350-8b7a27874cdc None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:10.892465 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f42e405-f89f-41f4-9350-8b7a27874cdc None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:10.892745 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f42e405-f89f-41f4-9350-8b7a27874cdc None admin] PATH_INFO: `/v3/projects/76848e532d98466e8520fa4e2fea27d1` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:10.893925 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f42e405-f89f-41f4-9350-8b7a27874cdc None admin] RBAC: Authorizing `identity:get_project(project_id=76848e532d98466e8520fa4e2fea27d1)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:10.895387 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f42e405-f89f-41f4-9350-8b7a27874cdc None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:10.897123 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 600/1184] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jun 7 14:22:10 2022] GET /identity/v3/projects/76848e532d98466e8520fa4e2fea27d1 => generated 361 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:22:10.903313 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd8b65e-d8df-4def-b3bd-e179e9653345 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:10.933643 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:11.006395 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd8b65e-d8df-4def-b3bd-e179e9653345 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:11.039624 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebdad73-3ea9-472e-8166-9f6043239335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:11.040281 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebdad73-3ea9-472e-8166-9f6043239335 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:11.040604 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebdad73-3ea9-472e-8166-9f6043239335 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:11.040883 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebdad73-3ea9-472e-8166-9f6043239335 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:11.041717 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebdad73-3ea9-472e-8166-9f6043239335 None admin] RBAC: Adding query filter params (role.id=9baff2b18f2649f7b757ff2e1f63c853, scope.project.id=76848e532d98466e8520fa4e2fea27d1, user.id=49414326a00b4a32a2d8aa68f68c9a83) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:22:11.042032 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebdad73-3ea9-472e-8166-9f6043239335 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:11.043469 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebdad73-3ea9-472e-8166-9f6043239335 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:11.049085 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 585/1185] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jun 7 14:22:10 2022] GET /identity/v3/role_assignments?user.id=49414326a00b4a32a2d8aa68f68c9a83&scope.project.id=76848e532d98466e8520fa4e2fea27d1&role.id=9baff2b18f2649f7b757ff2e1f63c853 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:22:13.294875 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e933592-13c7-4992-9a9f-a6f143d2fbca None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:13.295381 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e933592-13c7-4992-9a9f-a6f143d2fbca None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:13.295381 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e933592-13c7-4992-9a9f-a6f143d2fbca None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:13.296970 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 601/1186] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:22:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:22:13.302693 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e300694-b136-48e6-bf7c-674f490f4f9a None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:13.303175 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e300694-b136-48e6-bf7c-674f490f4f9a None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:13.303276 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e300694-b136-48e6-bf7c-674f490f4f9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:13.320866 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-8e300694-b136-48e6-bf7c-674f490f4f9a None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:22:13.322051 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-8e300694-b136-48e6-bf7c-674f490f4f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:13.389843 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 586/1187] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:22:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:22:13.412160 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e933592-13c7-4992-9a9f-a6f143d2fbca None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:13.487794 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:13.514684 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e933592-13c7-4992-9a9f-a6f143d2fbca None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:13.548407 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c72495e-f3ac-409c-8f56-8a5dc48b9d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:13.549113 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c72495e-f3ac-409c-8f56-8a5dc48b9d70 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:13.549397 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c72495e-f3ac-409c-8f56-8a5dc48b9d70 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:13.549671 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c72495e-f3ac-409c-8f56-8a5dc48b9d70 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:13.561324 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c72495e-f3ac-409c-8f56-8a5dc48b9d70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:22:13.569529 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-0c72495e-f3ac-409c-8f56-8a5dc48b9d70 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:22:13.569529 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:22:13.569529 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:22:13.569529 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:22:13.569529 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:22:13.569529 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:22:13.569529 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:22:13.569529 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:22:13.569529 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:22:13.569529 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:22:13.569529 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:22:13.569529 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:22:13.569529 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:22:13.569529 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:22:13.569529 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:22:13.569529 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:22:13.571557 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:22:13.571557 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:22:13.571557 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:22:13.571557 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:22:13.571557 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:22:13.571557 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:22:13.571557 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:22:13.571557 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:22:13.571557 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:22:13.571557 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:22:13.571557 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:22:13.571557 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:22:13.571557 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:22:13.571557 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:22:13.571557 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:22:13.571557 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:22:13.573224 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:22:13.573224 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:22:13.573224 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:22:13.573224 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:22:13.573224 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:22:13.573224 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:22:13.573224 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:22:13.573224 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:22:13.573224 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:22:13.573224 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:22:13.573224 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:22:13.573224 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:22:13.573224 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 602/1188] 10.222.0.85 () {56 vars in 1096 bytes} [Tue Jun 7 14:22:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:22:13.576840 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e300694-b136-48e6-bf7c-674f490f4f9a None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:13.601629 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:13.678957 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e300694-b136-48e6-bf7c-674f490f4f9a None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:13.713202 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516b0c1e-df5e-47ba-ac2c-8396949ec292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:13.713874 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516b0c1e-df5e-47ba-ac2c-8396949ec292 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:13.714163 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516b0c1e-df5e-47ba-ac2c-8396949ec292 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:13.714445 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516b0c1e-df5e-47ba-ac2c-8396949ec292 None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:13.715195 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516b0c1e-df5e-47ba-ac2c-8396949ec292 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:22:13.715505 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516b0c1e-df5e-47ba-ac2c-8396949ec292 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:13.716905 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516b0c1e-df5e-47ba-ac2c-8396949ec292 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:13.725699 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 587/1189] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jun 7 14:22:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:22:13.731643 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c72495e-f3ac-409c-8f56-8a5dc48b9d70 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:13.832854 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c72495e-f3ac-409c-8f56-8a5dc48b9d70 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:13.866698 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c939972b-c1ef-4b8a-b2b8-5d83ff11fe6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:13.867570 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c939972b-c1ef-4b8a-b2b8-5d83ff11fe6a None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:13.867856 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c939972b-c1ef-4b8a-b2b8-5d83ff11fe6a None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:13.868135 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c939972b-c1ef-4b8a-b2b8-5d83ff11fe6a None admin] PATH_INFO: `/v3/users/49414326a00b4a32a2d8aa68f68c9a83` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:13.869363 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c939972b-c1ef-4b8a-b2b8-5d83ff11fe6a None admin] RBAC: Authorizing `identity:get_user(user_id=49414326a00b4a32a2d8aa68f68c9a83)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:13.871063 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c939972b-c1ef-4b8a-b2b8-5d83ff11fe6a None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:13.872920 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 603/1190] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jun 7 14:22:13 2022] GET /identity/v3/users/49414326a00b4a32a2d8aa68f68c9a83 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:22:13.878898 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516b0c1e-df5e-47ba-ac2c-8396949ec292 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:13.963614 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:13.978752 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516b0c1e-df5e-47ba-ac2c-8396949ec292 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:14.013366 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a70e78-51c9-4ae2-b44a-7b8f969c96f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:14.014227 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a70e78-51c9-4ae2-b44a-7b8f969c96f6 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:14.014640 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a70e78-51c9-4ae2-b44a-7b8f969c96f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:14.014793 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a70e78-51c9-4ae2-b44a-7b8f969c96f6 None admin] PATH_INFO: `/v3/projects/76848e532d98466e8520fa4e2fea27d1` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:14.015986 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a70e78-51c9-4ae2-b44a-7b8f969c96f6 None admin] RBAC: Authorizing `identity:get_project(project_id=76848e532d98466e8520fa4e2fea27d1)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:14.017471 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a70e78-51c9-4ae2-b44a-7b8f969c96f6 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:14.019160 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 588/1191] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jun 7 14:22:13 2022] GET /identity/v3/projects/76848e532d98466e8520fa4e2fea27d1 => generated 361 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:22:14.025766 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c939972b-c1ef-4b8a-b2b8-5d83ff11fe6a None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:14.028700 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:14.126690 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c939972b-c1ef-4b8a-b2b8-5d83ff11fe6a None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:14.162450 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c1a0d0-d05c-4df9-aa2b-a3e7e3175a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:14.163332 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c1a0d0-d05c-4df9-aa2b-a3e7e3175a59 None admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:14.163617 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c1a0d0-d05c-4df9-aa2b-a3e7e3175a59 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:14.163897 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c1a0d0-d05c-4df9-aa2b-a3e7e3175a59 None admin] PATH_INFO: `/v3/projects/76848e532d98466e8520fa4e2fea27d1/users/49414326a00b4a32a2d8aa68f68c9a83/roles/9baff2b18f2649f7b757ff2e1f63c853` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:14.165911 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c1a0d0-d05c-4df9-aa2b-a3e7e3175a59 None admin] RBAC: Authorizing `identity:create_grant(project_id=76848e532d98466e8520fa4e2fea27d1, user_id=49414326a00b4a32a2d8aa68f68c9a83, role_id=9baff2b18f2649f7b757ff2e1f63c853)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:14.167334 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c1a0d0-d05c-4df9-aa2b-a3e7e3175a59 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:14.230218 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 604/1192] 10.222.0.85 () {58 vars in 1439 bytes} [Tue Jun 7 14:22:14 2022] PUT /identity/v3/projects/76848e532d98466e8520fa4e2fea27d1/users/49414326a00b4a32a2d8aa68f68c9a83/roles/9baff2b18f2649f7b757ff2e1f63c853 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:22:16.543178 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d97827-7b32-4553-9300-67130b9794ad None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:16.543711 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d97827-7b32-4553-9300-67130b9794ad None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:16.543827 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d97827-7b32-4553-9300-67130b9794ad None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:16.545153 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 589/1193] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:22:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:22:16.551079 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef4d8eb-d2ec-44e1-815b-039280cd5288 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:16.551471 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef4d8eb-d2ec-44e1-815b-039280cd5288 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:16.551647 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef4d8eb-d2ec-44e1-815b-039280cd5288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:16.568678 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-2ef4d8eb-d2ec-44e1-815b-039280cd5288 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:22:16.569875 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-2ef4d8eb-d2ec-44e1-815b-039280cd5288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:22:16.626753 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:16.637612 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 605/1194] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:22:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:22:16.661534 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d97827-7b32-4553-9300-67130b9794ad None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:16.766505 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d97827-7b32-4553-9300-67130b9794ad None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:16.801341 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a828f9c5-4dff-4d7c-acd4-65f3c6309ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:16.802077 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a828f9c5-4dff-4d7c-acd4-65f3c6309ef2 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:16.802360 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a828f9c5-4dff-4d7c-acd4-65f3c6309ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:16.802658 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a828f9c5-4dff-4d7c-acd4-65f3c6309ef2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:16.814977 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a828f9c5-4dff-4d7c-acd4-65f3c6309ef2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:22:16.822786 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-a828f9c5-4dff-4d7c-acd4-65f3c6309ef2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:22:16.822786 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:22:16.822786 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:22:16.822786 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:22:16.822786 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:22:16.822786 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:22:16.822786 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:22:16.822786 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:22:16.822786 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:22:16.822786 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:22:16.822786 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:22:16.822786 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:22:16.822786 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:22:16.822786 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:22:16.822786 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:22:16.822786 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:22:16.823897 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:22:16.823897 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:22:16.823897 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:22:16.823897 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:22:16.823897 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:22:16.823897 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:22:16.823897 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:22:16.823897 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:22:16.823897 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:22:16.823897 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:22:16.823897 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:22:16.823897 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:22:16.823897 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:22:16.823897 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:22:16.823897 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:22:16.823897 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:22:16.824853 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:22:16.824853 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:22:16.824853 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:22:16.824853 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:22:16.824853 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:22:16.824853 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:22:16.824853 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:22:16.824853 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:22:16.824853 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:22:16.824853 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:22:16.824853 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:22:16.824853 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:22:16.824853 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 590/1195] 10.222.0.85 () {56 vars in 1096 bytes} [Tue Jun 7 14:22:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:22:16.830250 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef4d8eb-d2ec-44e1-815b-039280cd5288 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:16.929730 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef4d8eb-d2ec-44e1-815b-039280cd5288 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:16.964137 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feaee92-67ab-4717-9640-69952fe49d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:16.964859 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feaee92-67ab-4717-9640-69952fe49d6f None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:16.965160 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feaee92-67ab-4717-9640-69952fe49d6f None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:16.965450 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feaee92-67ab-4717-9640-69952fe49d6f None admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:16.966232 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6feaee92-67ab-4717-9640-69952fe49d6f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:22:16.966536 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6feaee92-67ab-4717-9640-69952fe49d6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:16.967957 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6feaee92-67ab-4717-9640-69952fe49d6f None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:16.977651 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 606/1196] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jun 7 14:22:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:22:16.983520 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a828f9c5-4dff-4d7c-acd4-65f3c6309ef2 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:17.000001 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:17.087433 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a828f9c5-4dff-4d7c-acd4-65f3c6309ef2 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:17.122012 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6b5d8d-2929-4448-a491-56a118255459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:17.122877 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6b5d8d-2929-4448-a491-56a118255459 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:17.123165 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6b5d8d-2929-4448-a491-56a118255459 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:17.123448 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6b5d8d-2929-4448-a491-56a118255459 None admin] PATH_INFO: `/v3/users/49414326a00b4a32a2d8aa68f68c9a83` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:17.124680 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6b5d8d-2929-4448-a491-56a118255459 None admin] RBAC: Authorizing `identity:get_user(user_id=49414326a00b4a32a2d8aa68f68c9a83)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:17.126111 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6b5d8d-2929-4448-a491-56a118255459 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:17.127740 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 591/1197] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jun 7 14:22:16 2022] GET /identity/v3/users/49414326a00b4a32a2d8aa68f68c9a83 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:22:17.133737 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feaee92-67ab-4717-9640-69952fe49d6f None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:17.191951 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:17.237076 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feaee92-67ab-4717-9640-69952fe49d6f None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:17.270424 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c1321b-55d8-42c4-af76-60f31d543cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:17.271072 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c1321b-55d8-42c4-af76-60f31d543cb4 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:17.271361 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c1321b-55d8-42c4-af76-60f31d543cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:17.271642 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c1321b-55d8-42c4-af76-60f31d543cb4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:17.341651 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c1321b-55d8-42c4-af76-60f31d543cb4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:17.343101 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c1321b-55d8-42c4-af76-60f31d543cb4 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:17.399322 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:17.446602 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 607/1198] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jun 7 14:22:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 14:22:17.452517 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6b5d8d-2929-4448-a491-56a118255459 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:17.532596 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:17.551029 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6b5d8d-2929-4448-a491-56a118255459 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:17.584269 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c90782-2b71-4f01-8c2b-472df8c32801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:17.585151 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c90782-2b71-4f01-8c2b-472df8c32801 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:17.585434 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c90782-2b71-4f01-8c2b-472df8c32801 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:17.585708 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c90782-2b71-4f01-8c2b-472df8c32801 None admin] PATH_INFO: `/v3/projects/76848e532d98466e8520fa4e2fea27d1` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:17.586895 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c90782-2b71-4f01-8c2b-472df8c32801 None admin] RBAC: Authorizing `identity:get_project(project_id=76848e532d98466e8520fa4e2fea27d1)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:17.588331 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c90782-2b71-4f01-8c2b-472df8c32801 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:17.590046 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 592/1199] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jun 7 14:22:17 2022] GET /identity/v3/projects/76848e532d98466e8520fa4e2fea27d1 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:22:17.596261 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c1321b-55d8-42c4-af76-60f31d543cb4 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:22:17.698252 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c1321b-55d8-42c4-af76-60f31d543cb4 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:22:17.731722 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f0465d-c5ad-4e3d-9b0e-533d070f6880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:22:17.732397 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f0465d-c5ad-4e3d-9b0e-533d070f6880 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:22:17.732693 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f0465d-c5ad-4e3d-9b0e-533d070f6880 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:22:17.732973 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f0465d-c5ad-4e3d-9b0e-533d070f6880 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:22:17.733803 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f0465d-c5ad-4e3d-9b0e-533d070f6880 None admin] RBAC: Adding query filter params (role.id=9baff2b18f2649f7b757ff2e1f63c853, scope.project.id=76848e532d98466e8520fa4e2fea27d1, user.id=49414326a00b4a32a2d8aa68f68c9a83) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:22:17.734112 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f0465d-c5ad-4e3d-9b0e-533d070f6880 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:22:17.735533 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f0465d-c5ad-4e3d-9b0e-533d070f6880 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:22:17.741909 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 608/1200] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jun 7 14:22:17 2022] GET /identity/v3/role_assignments?user.id=49414326a00b4a32a2d8aa68f68c9a83&scope.project.id=76848e532d98466e8520fa4e2fea27d1&role.id=9baff2b18f2649f7b757ff2e1f63c853 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:22:27.579813 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=25 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:22:27.739246 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=91 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:37.913376 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9a92ca-3fea-491c-9842-f2db9f66f962 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:37.914000 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9a92ca-3fea-491c-9842-f2db9f66f962 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:37.914000 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9a92ca-3fea-491c-9842-f2db9f66f962 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:37.915336 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 593/1201] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:23:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:23:37.920520 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db524deb-d6f6-4c0a-9c0c-f5ed56329012 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:37.920943 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db524deb-d6f6-4c0a-9c0c-f5ed56329012 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:37.921095 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db524deb-d6f6-4c0a-9c0c-f5ed56329012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:37.938952 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-db524deb-d6f6-4c0a-9c0c-f5ed56329012 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:23:37.940205 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-db524deb-d6f6-4c0a-9c0c-f5ed56329012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:38.007825 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 609/1202] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:23:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:23:38.032758 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9a92ca-3fea-491c-9842-f2db9f66f962 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:38.134770 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9a92ca-3fea-491c-9842-f2db9f66f962 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:38.169228 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9567c713-44c8-4c4f-94a9-1698dc2fc578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:38.169941 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9567c713-44c8-4c4f-94a9-1698dc2fc578 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:38.170228 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9567c713-44c8-4c4f-94a9-1698dc2fc578 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:38.170508 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9567c713-44c8-4c4f-94a9-1698dc2fc578 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:38.180392 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9567c713-44c8-4c4f-94a9-1698dc2fc578 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:38.181857 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9567c713-44c8-4c4f-94a9-1698dc2fc578 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:38.201661 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-9567c713-44c8-4c4f-94a9-1698dc2fc578 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:23:38.201661 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:23:38.201661 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:23:38.201661 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:23:38.201661 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:23:38.201661 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:23:38.201661 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:23:38.201661 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:23:38.201661 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:23:38.201661 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:23:38.201661 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:23:38.201661 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:23:38.201661 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:23:38.201661 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:23:38.201661 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:23:38.203517 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:23:38.203517 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:23:38.203517 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:23:38.203517 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:23:38.203517 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:23:38.203517 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:23:38.203517 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:23:38.203517 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:23:38.203517 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:23:38.203517 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:23:38.203517 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:23:38.203517 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:23:38.203517 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:23:38.203517 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:23:38.203517 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:23:38.203517 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:23:38.203517 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:23:38.205539 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:23:38.205539 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:23:38.205539 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:23:38.205539 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:23:38.205539 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 594/1203] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:23:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:23:38.210011 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db524deb-d6f6-4c0a-9c0c-f5ed56329012 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:38.311036 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db524deb-d6f6-4c0a-9c0c-f5ed56329012 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:38.339939 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:38.345663 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1b319a-e793-4eb4-9bea-fd2531465bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:38.346365 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1b319a-e793-4eb4-9bea-fd2531465bd9 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:38.346656 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1b319a-e793-4eb4-9bea-fd2531465bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:38.346931 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1b319a-e793-4eb4-9bea-fd2531465bd9 None admin] PATH_INFO: `/v3/domains` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:38.347654 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1b319a-e793-4eb4-9bea-fd2531465bd9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:23:38.347963 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1b319a-e793-4eb4-9bea-fd2531465bd9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:38.349358 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1b319a-e793-4eb4-9bea-fd2531465bd9 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:38.360569 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 610/1204] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jun 7 14:23:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:23:38.366324 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9567c713-44c8-4c4f-94a9-1698dc2fc578 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:38.425376 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:38.472213 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9567c713-44c8-4c4f-94a9-1698dc2fc578 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:38.505800 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478cde33-b849-480d-8f66-f3acd9e63c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:38.506467 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478cde33-b849-480d-8f66-f3acd9e63c0d None admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:38.506747 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478cde33-b849-480d-8f66-f3acd9e63c0d None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:38.507024 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478cde33-b849-480d-8f66-f3acd9e63c0d None admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:38.507730 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478cde33-b849-480d-8f66-f3acd9e63c0d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:38.509686 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478cde33-b849-480d-8f66-f3acd9e63c0d None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:38.657060 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 595/1205] 10.222.0.85 () {60 vars in 1132 bytes} [Tue Jun 7 14:23:38 2022] POST /identity/v3/users => generated 251 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:23:40.949552 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525c89bc-3957-426c-b8dc-15cf0438b364 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:40.950119 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525c89bc-3957-426c-b8dc-15cf0438b364 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:40.950226 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525c89bc-3957-426c-b8dc-15cf0438b364 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:40.951569 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 611/1206] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:23:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:23:40.956877 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d76f94-f8ef-4faa-91ad-d890832c7779 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:40.957341 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d76f94-f8ef-4faa-91ad-d890832c7779 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:40.957458 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d76f94-f8ef-4faa-91ad-d890832c7779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:40.974668 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-38d76f94-f8ef-4faa-91ad-d890832c7779 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:23:40.975841 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-38d76f94-f8ef-4faa-91ad-d890832c7779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:41.041736 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 596/1207] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:23:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:23:41.065640 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525c89bc-3957-426c-b8dc-15cf0438b364 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:41.166466 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525c89bc-3957-426c-b8dc-15cf0438b364 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:41.200673 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129ec52e-9fd0-4c70-b403-cdf0041e36e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:41.201393 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129ec52e-9fd0-4c70-b403-cdf0041e36e3 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:41.201672 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129ec52e-9fd0-4c70-b403-cdf0041e36e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:41.201951 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129ec52e-9fd0-4c70-b403-cdf0041e36e3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:41.212527 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129ec52e-9fd0-4c70-b403-cdf0041e36e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:23:41.220180 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-129ec52e-9fd0-4c70-b403-cdf0041e36e3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:23:41.220180 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:23:41.220180 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:23:41.220180 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:23:41.220180 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:23:41.220180 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:23:41.220180 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:23:41.220180 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:23:41.220180 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:23:41.220180 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:23:41.220180 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:23:41.220180 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:23:41.220180 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:23:41.220180 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:23:41.220180 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:23:41.220180 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:23:41.221185 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:23:41.221185 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:23:41.221185 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:23:41.221185 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:23:41.221185 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:23:41.221185 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:23:41.221185 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:23:41.221185 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:23:41.221185 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:23:41.221185 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:23:41.221185 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:23:41.221185 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:23:41.221185 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:23:41.221185 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:23:41.221185 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:23:41.221185 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:23:41.222076 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:23:41.222076 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:23:41.222076 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:23:41.222076 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:23:41.222076 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:23:41.222076 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:23:41.222076 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:23:41.222076 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:23:41.222076 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:23:41.222076 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:23:41.222076 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:23:41.222076 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:23:41.222076 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 612/1208] 10.222.0.85 () {56 vars in 1102 bytes} [Tue Jun 7 14:23:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:23:41.227589 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d76f94-f8ef-4faa-91ad-d890832c7779 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:41.266651 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:41.324337 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d76f94-f8ef-4faa-91ad-d890832c7779 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:41.357753 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6ede0a-f24c-4986-a34b-9a5568588490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:41.358240 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6ede0a-f24c-4986-a34b-9a5568588490 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:41.358524 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6ede0a-f24c-4986-a34b-9a5568588490 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:41.358803 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6ede0a-f24c-4986-a34b-9a5568588490 None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:41.359546 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6ede0a-f24c-4986-a34b-9a5568588490 None admin] RBAC: Adding query filter params (name=service) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:23:41.359855 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6ede0a-f24c-4986-a34b-9a5568588490 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:41.361303 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6ede0a-f24c-4986-a34b-9a5568588490 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:41.369227 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 597/1209] 10.222.0.85 () {56 vars in 1103 bytes} [Tue Jun 7 14:23:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:23:41.375181 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129ec52e-9fd0-4c70-b403-cdf0041e36e3 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:41.425534 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:41.473972 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129ec52e-9fd0-4c70-b403-cdf0041e36e3 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:41.509217 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43146436-5cff-424f-a2cb-38a1372465d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:41.510004 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43146436-5cff-424f-a2cb-38a1372465d7 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:41.510291 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43146436-5cff-424f-a2cb-38a1372465d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:41.510569 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43146436-5cff-424f-a2cb-38a1372465d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:41.520902 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43146436-5cff-424f-a2cb-38a1372465d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:41.522342 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43146436-5cff-424f-a2cb-38a1372465d7 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:41.539965 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-43146436-5cff-424f-a2cb-38a1372465d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:23:41.539965 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:23:41.539965 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:23:41.539965 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:23:41.539965 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:23:41.539965 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:23:41.539965 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:23:41.539965 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:23:41.539965 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:23:41.539965 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:23:41.539965 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:23:41.539965 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:23:41.539965 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:23:41.539965 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:23:41.539965 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:23:41.541097 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:23:41.541097 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:23:41.541097 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:23:41.541097 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:23:41.541097 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:23:41.541097 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:23:41.541097 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:23:41.541097 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:23:41.541097 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:23:41.541097 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:23:41.541097 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:23:41.541097 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:23:41.541097 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:23:41.541097 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:23:41.541097 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:23:41.541097 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:23:41.541097 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:23:41.542340 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:23:41.542340 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:23:41.542340 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:23:41.542340 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:23:41.542340 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 613/1210] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:23:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:23:41.547044 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6ede0a-f24c-4986-a34b-9a5568588490 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:41.645222 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:41.648316 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6ede0a-f24c-4986-a34b-9a5568588490 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:41.682617 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f7b4d8-f013-4c6e-bfba-ebb649ce09dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:41.683274 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f7b4d8-f013-4c6e-bfba-ebb649ce09dd None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:41.683558 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f7b4d8-f013-4c6e-bfba-ebb649ce09dd None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:41.683832 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f7b4d8-f013-4c6e-bfba-ebb649ce09dd None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:41.684571 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f7b4d8-f013-4c6e-bfba-ebb649ce09dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:23:41.684890 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f7b4d8-f013-4c6e-bfba-ebb649ce09dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:41.686242 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f7b4d8-f013-4c6e-bfba-ebb649ce09dd None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:41.697643 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 598/1211] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jun 7 14:23:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:23:41.703269 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43146436-5cff-424f-a2cb-38a1372465d7 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:41.805317 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43146436-5cff-424f-a2cb-38a1372465d7 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:41.806812 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:41.841654 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c571cde1-a4e4-484a-be60-84bb0421dbc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:41.842477 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c571cde1-a4e4-484a-be60-84bb0421dbc7 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:41.842760 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c571cde1-a4e4-484a-be60-84bb0421dbc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:41.843033 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c571cde1-a4e4-484a-be60-84bb0421dbc7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:41.849975 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c571cde1-a4e4-484a-be60-84bb0421dbc7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:41.851391 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c571cde1-a4e4-484a-be60-84bb0421dbc7 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:41.866214 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-c571cde1-a4e4-484a-be60-84bb0421dbc7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 07 14:23:41.866214 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:23:41.866214 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:23:41.866214 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:23:41.866214 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:23:41.866214 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:23:41.866214 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:23:41.866214 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:23:41.866214 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:23:41.866214 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:23:41.866214 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:23:41.866214 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:23:41.866214 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 14:23:41.866214 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 14:23:41.866214 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 14:23:41.866214 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 14:23:41.867281 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:23:41.867281 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:23:41.867281 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 14:23:41.867281 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:23:41.867281 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 14:23:41.867281 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:23:41.867281 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:23:41.867281 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:23:41.867281 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:23:41.867281 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:23:41.867281 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:23:41.867281 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:23:41.867281 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:23:41.867281 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:23:41.867281 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:23:41.867281 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:23:41.867281 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:23:41.867281 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:23:41.868241 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:23:41.868241 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:23:41.868241 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 14:23:41.868241 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 14:23:41.868241 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 14:23:41.868241 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 14:23:41.868241 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 14:23:41.868241 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 14:23:41.868241 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 07 14:23:41.868241 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:23:41.868241 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 614/1212] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:23:41 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:23:41.873708 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f7b4d8-f013-4c6e-bfba-ebb649ce09dd None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:41.975361 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f7b4d8-f013-4c6e-bfba-ebb649ce09dd None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:42.010038 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f008bb06-c41e-475b-bc05-1b2bb53f24dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:42.010720 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f008bb06-c41e-475b-bc05-1b2bb53f24dd None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:42.011068 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f008bb06-c41e-475b-bc05-1b2bb53f24dd None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:42.011351 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f008bb06-c41e-475b-bc05-1b2bb53f24dd None admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:42.012206 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f008bb06-c41e-475b-bc05-1b2bb53f24dd None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:23:42.012547 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f008bb06-c41e-475b-bc05-1b2bb53f24dd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:42.013995 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f008bb06-c41e-475b-bc05-1b2bb53f24dd None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:42.031672 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 599/1213] 10.222.0.85 () {56 vars in 1143 bytes} [Tue Jun 7 14:23:41 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:23:42.037444 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c571cde1-a4e4-484a-be60-84bb0421dbc7 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:42.143167 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c571cde1-a4e4-484a-be60-84bb0421dbc7 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:42.175939 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d6ef32-87b5-4fa4-a7b6-abd43bfe8e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:42.176612 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d6ef32-87b5-4fa4-a7b6-abd43bfe8e1f None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:42.176896 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d6ef32-87b5-4fa4-a7b6-abd43bfe8e1f None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:42.177170 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d6ef32-87b5-4fa4-a7b6-abd43bfe8e1f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:42.205590 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:42.247431 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d6ef32-87b5-4fa4-a7b6-abd43bfe8e1f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:42.248920 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d6ef32-87b5-4fa4-a7b6-abd43bfe8e1f None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:42.345295 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 615/1214] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jun 7 14:23:42 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 14:23:42.351052 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f008bb06-c41e-475b-bc05-1b2bb53f24dd None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:42.369953 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:42.448033 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f008bb06-c41e-475b-bc05-1b2bb53f24dd None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:42.482441 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76a9890-571c-4332-91e1-5f87cd130646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:42.483222 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76a9890-571c-4332-91e1-5f87cd130646 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:42.483501 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76a9890-571c-4332-91e1-5f87cd130646 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:42.483770 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76a9890-571c-4332-91e1-5f87cd130646 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:42.494261 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76a9890-571c-4332-91e1-5f87cd130646 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:42.495642 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76a9890-571c-4332-91e1-5f87cd130646 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:42.513370 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-a76a9890-571c-4332-91e1-5f87cd130646 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:23:42.513370 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:23:42.513370 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:23:42.513370 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:23:42.513370 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:23:42.513370 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:23:42.513370 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:23:42.513370 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:23:42.513370 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:23:42.513370 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:23:42.513370 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:23:42.513370 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:23:42.513370 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:23:42.513370 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:23:42.513370 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:23:42.514484 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:23:42.514484 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:23:42.514484 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:23:42.514484 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:23:42.514484 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:23:42.514484 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:23:42.514484 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:23:42.514484 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:23:42.514484 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:23:42.514484 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:23:42.514484 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:23:42.514484 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:23:42.514484 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:23:42.514484 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:23:42.514484 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:23:42.514484 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:23:42.514484 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:23:42.515598 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:23:42.515598 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:23:42.515598 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:23:42.515598 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:23:42.515598 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 600/1215] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:23:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:23:42.520439 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d6ef32-87b5-4fa4-a7b6-abd43bfe8e1f None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:42.582217 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:42.621394 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d6ef32-87b5-4fa4-a7b6-abd43bfe8e1f None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:42.655675 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2cb8bc-10f7-4d1c-a385-b20a0ece8f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:42.656337 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2cb8bc-10f7-4d1c-a385-b20a0ece8f6e None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:42.656706 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2cb8bc-10f7-4d1c-a385-b20a0ece8f6e None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:42.656990 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2cb8bc-10f7-4d1c-a385-b20a0ece8f6e None admin] PATH_INFO: `/v3/domains` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:42.657715 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2cb8bc-10f7-4d1c-a385-b20a0ece8f6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:23:42.658029 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2cb8bc-10f7-4d1c-a385-b20a0ece8f6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:42.659446 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2cb8bc-10f7-4d1c-a385-b20a0ece8f6e None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:42.672967 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 616/1216] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jun 7 14:23:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:23:42.678708 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76a9890-571c-4332-91e1-5f87cd130646 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:42.748459 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:42.779449 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76a9890-571c-4332-91e1-5f87cd130646 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:42.812059 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c44ea52-c1b5-41b6-b8cf-0ff19b84b5af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:42.812952 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c44ea52-c1b5-41b6-b8cf-0ff19b84b5af None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:42.813242 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c44ea52-c1b5-41b6-b8cf-0ff19b84b5af None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:42.813523 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c44ea52-c1b5-41b6-b8cf-0ff19b84b5af None admin] PATH_INFO: `/v3/projects/service` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:42.820258 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c44ea52-c1b5-41b6-b8cf-0ff19b84b5af None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:42.823998 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c44ea52-c1b5-41b6-b8cf-0ff19b84b5af None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:42.838873 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-8c44ea52-c1b5-41b6-b8cf-0ff19b84b5af None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:23:42.838873 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:23:42.838873 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:23:42.838873 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:23:42.838873 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:23:42.838873 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:23:42.838873 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:23:42.838873 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:23:42.838873 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:23:42.838873 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:23:42.838873 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:23:42.838873 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:23:42.838873 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 14:23:42.838873 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 14:23:42.838873 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 14:23:42.840004 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 14:23:42.840004 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:23:42.840004 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:23:42.840004 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:23:42.840004 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:23:42.840004 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:23:42.840004 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:23:42.840004 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:23:42.840004 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:23:42.840004 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:23:42.840004 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:23:42.840004 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:23:42.840004 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:23:42.840004 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:23:42.840004 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:23:42.840004 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:23:42.840004 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:23:42.841106 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 14:23:42.841106 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 14:23:42.841106 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 14:23:42.841106 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 14:23:42.841106 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 14:23:42.841106 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 14:23:42.841106 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:23:42.841106 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:23:42.841106 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 601/1217] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jun 7 14:23:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:23:42.846089 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2cb8bc-10f7-4d1c-a385-b20a0ece8f6e None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:42.944829 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2cb8bc-10f7-4d1c-a385-b20a0ece8f6e None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:42.969221 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:42.980761 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0a6fed-35b3-493f-bcaa-1bb980bba594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:42.981428 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0a6fed-35b3-493f-bcaa-1bb980bba594 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:42.981719 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0a6fed-35b3-493f-bcaa-1bb980bba594 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:42.981995 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0a6fed-35b3-493f-bcaa-1bb980bba594 None admin] PATH_INFO: `/v3/projects` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:42.982783 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0a6fed-35b3-493f-bcaa-1bb980bba594 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:23:42.983096 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0a6fed-35b3-493f-bcaa-1bb980bba594 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:42.984553 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0a6fed-35b3-493f-bcaa-1bb980bba594 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:42.995887 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 617/1218] 10.222.0.85 () {56 vars in 1148 bytes} [Tue Jun 7 14:23:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:23:43.002310 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c44ea52-c1b5-41b6-b8cf-0ff19b84b5af None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:43.102370 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c44ea52-c1b5-41b6-b8cf-0ff19b84b5af None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:43.130955 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:43.137598 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01fde30-f678-4612-9606-2c1da0346f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:43.138331 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01fde30-f678-4612-9606-2c1da0346f01 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:43.138692 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01fde30-f678-4612-9606-2c1da0346f01 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:43.138912 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01fde30-f678-4612-9606-2c1da0346f01 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:43.139790 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01fde30-f678-4612-9606-2c1da0346f01 None admin] RBAC: Adding query filter params (role.id=17516844e8b947b389d62f50718b852d, scope.project.id=724785eb74e44daf8736fd01fc0d1839, user.id=6330294b1c8e4dcd85aad189f007b0aa) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:23:43.140107 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01fde30-f678-4612-9606-2c1da0346f01 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:43.141590 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01fde30-f678-4612-9606-2c1da0346f01 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:43.147707 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 602/1219] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jun 7 14:23:43 2022] GET /identity/v3/role_assignments?user.id=6330294b1c8e4dcd85aad189f007b0aa&scope.project.id=724785eb74e44daf8736fd01fc0d1839&role.id=17516844e8b947b389d62f50718b852d => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:23:45.387410 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1417645a-57c2-479c-817d-6d9e4338eb10 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:45.387934 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1417645a-57c2-479c-817d-6d9e4338eb10 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:45.387934 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1417645a-57c2-479c-817d-6d9e4338eb10 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:45.389308 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 618/1220] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:23:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:23:45.394712 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f9463-f78b-48dc-9077-1984f124c8a3 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:45.395094 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f9463-f78b-48dc-9077-1984f124c8a3 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:45.395284 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f9463-f78b-48dc-9077-1984f124c8a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:45.412180 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-f80f9463-f78b-48dc-9077-1984f124c8a3 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:23:45.413392 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-f80f9463-f78b-48dc-9077-1984f124c8a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:45.481741 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 603/1221] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:23:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 14:23:45.505112 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1417645a-57c2-479c-817d-6d9e4338eb10 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:45.609991 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1417645a-57c2-479c-817d-6d9e4338eb10 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:45.644517 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9488f685-0e5b-4dd8-b7fb-17666c90aafa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:45.645238 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9488f685-0e5b-4dd8-b7fb-17666c90aafa None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:45.645525 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9488f685-0e5b-4dd8-b7fb-17666c90aafa None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:45.645804 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9488f685-0e5b-4dd8-b7fb-17666c90aafa None admin] PATH_INFO: `/v3/roles/service` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:45.656479 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9488f685-0e5b-4dd8-b7fb-17666c90aafa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:23:45.664230 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-9488f685-0e5b-4dd8-b7fb-17666c90aafa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:23:45.664230 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:23:45.664230 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:23:45.664230 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:23:45.664230 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:23:45.664230 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:23:45.664230 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:23:45.664230 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:23:45.664230 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:23:45.664230 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:23:45.664230 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:23:45.664230 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:23:45.664230 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:23:45.664230 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:23:45.664230 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:23:45.664230 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:23:45.666070 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:23:45.666070 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:23:45.666070 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:23:45.666070 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:23:45.666070 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:23:45.666070 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:23:45.666070 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:23:45.666070 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:23:45.666070 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:23:45.666070 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:23:45.666070 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:23:45.666070 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:23:45.666070 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:23:45.666070 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:23:45.666070 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:23:45.666070 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:23:45.667320 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:23:45.667320 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:23:45.667320 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:23:45.667320 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:23:45.667320 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:23:45.667320 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:23:45.667320 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:23:45.667320 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:23:45.667320 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:23:45.667320 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:23:45.667320 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:23:45.667320 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:23:45.667320 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 619/1222] 10.222.0.85 () {56 vars in 1102 bytes} [Tue Jun 7 14:23:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:23:45.672260 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80f9463-f78b-48dc-9077-1984f124c8a3 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:45.771180 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80f9463-f78b-48dc-9077-1984f124c8a3 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:45.794604 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:45.805004 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f699b726-3b4e-4ac8-9c4b-b0383ee66fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:45.805681 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f699b726-3b4e-4ac8-9c4b-b0383ee66fcc None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:45.805963 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f699b726-3b4e-4ac8-9c4b-b0383ee66fcc None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:45.806239 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f699b726-3b4e-4ac8-9c4b-b0383ee66fcc None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:45.806993 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f699b726-3b4e-4ac8-9c4b-b0383ee66fcc None admin] RBAC: Adding query filter params (name=service) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:23:45.807301 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f699b726-3b4e-4ac8-9c4b-b0383ee66fcc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:45.808746 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f699b726-3b4e-4ac8-9c4b-b0383ee66fcc None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:45.820990 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 604/1223] 10.222.0.85 () {56 vars in 1103 bytes} [Tue Jun 7 14:23:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:23:45.830351 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9488f685-0e5b-4dd8-b7fb-17666c90aafa None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:45.875490 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:45.931718 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9488f685-0e5b-4dd8-b7fb-17666c90aafa None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:45.965522 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9240ccf8-4f7a-4e5d-9d4b-41a2affd6e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:45.966316 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9240ccf8-4f7a-4e5d-9d4b-41a2affd6e4f None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:45.966604 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9240ccf8-4f7a-4e5d-9d4b-41a2affd6e4f None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:45.966884 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9240ccf8-4f7a-4e5d-9d4b-41a2affd6e4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:45.978316 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9240ccf8-4f7a-4e5d-9d4b-41a2affd6e4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:45.981393 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9240ccf8-4f7a-4e5d-9d4b-41a2affd6e4f None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:45.998434 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-9240ccf8-4f7a-4e5d-9d4b-41a2affd6e4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:23:45.998434 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:23:45.998434 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:23:45.998434 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:23:45.998434 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:23:45.998434 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:23:45.998434 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:23:45.998434 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:23:45.998434 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:23:45.998434 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:23:45.998434 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:23:45.998434 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:23:45.998434 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:23:45.998434 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:23:45.998434 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:23:45.999545 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:23:45.999545 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:23:45.999545 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:23:45.999545 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:23:45.999545 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:23:45.999545 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:23:45.999545 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:23:45.999545 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:23:45.999545 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:23:45.999545 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:23:45.999545 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:23:45.999545 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:23:45.999545 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:23:45.999545 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:23:45.999545 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:23:45.999545 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:23:45.999545 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:23:46.000761 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:23:46.000761 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:23:46.000761 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:23:46.000761 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:23:46.000761 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 620/1224] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:23:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:23:46.005580 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f699b726-3b4e-4ac8-9c4b-b0383ee66fcc None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:46.104590 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f699b726-3b4e-4ac8-9c4b-b0383ee66fcc None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:46.140818 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154c88da-9fa0-4697-9e62-4e09f681a2ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:46.141480 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154c88da-9fa0-4697-9e62-4e09f681a2ad None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:46.141764 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154c88da-9fa0-4697-9e62-4e09f681a2ad None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:46.142050 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154c88da-9fa0-4697-9e62-4e09f681a2ad None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:46.142825 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154c88da-9fa0-4697-9e62-4e09f681a2ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:23:46.143134 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154c88da-9fa0-4697-9e62-4e09f681a2ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:46.144520 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154c88da-9fa0-4697-9e62-4e09f681a2ad None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:46.155974 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 605/1225] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jun 7 14:23:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:23:46.161688 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9240ccf8-4f7a-4e5d-9d4b-41a2affd6e4f None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:46.254966 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:46.262117 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9240ccf8-4f7a-4e5d-9d4b-41a2affd6e4f None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:46.297602 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1873c80-c649-49af-a8a8-9c5d6d0c48f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:46.298425 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1873c80-c649-49af-a8a8-9c5d6d0c48f7 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:46.298739 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1873c80-c649-49af-a8a8-9c5d6d0c48f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:46.299016 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1873c80-c649-49af-a8a8-9c5d6d0c48f7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:46.311375 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1873c80-c649-49af-a8a8-9c5d6d0c48f7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:46.312876 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1873c80-c649-49af-a8a8-9c5d6d0c48f7 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:46.328455 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-f1873c80-c649-49af-a8a8-9c5d6d0c48f7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 07 14:23:46.328455 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:23:46.328455 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:23:46.328455 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:23:46.328455 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:23:46.328455 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:23:46.328455 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:23:46.328455 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:23:46.328455 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:23:46.328455 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:23:46.328455 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:23:46.328455 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:23:46.328455 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 14:23:46.328455 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 14:23:46.328455 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 14:23:46.328455 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 14:23:46.329710 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:23:46.329710 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:23:46.329710 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 14:23:46.329710 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:23:46.329710 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 14:23:46.329710 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:23:46.329710 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:23:46.329710 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:23:46.329710 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:23:46.329710 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:23:46.329710 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:23:46.329710 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:23:46.329710 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:23:46.329710 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:23:46.329710 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:23:46.329710 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:23:46.329710 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:23:46.329710 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:23:46.330675 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:23:46.330675 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:23:46.330675 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 14:23:46.330675 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 14:23:46.330675 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 14:23:46.330675 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 14:23:46.330675 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 14:23:46.330675 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 14:23:46.330675 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 07 14:23:46.330675 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:23:46.330675 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 621/1226] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:23:46 2022] GET /identity/v3/users/placement => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:23:46.335543 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154c88da-9fa0-4697-9e62-4e09f681a2ad None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:46.382149 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:46.434276 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154c88da-9fa0-4697-9e62-4e09f681a2ad None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:46.468543 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec0892f-d8e7-4066-9c52-b42d82ac8ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:46.469210 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0892f-d8e7-4066-9c52-b42d82ac8ae1 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:46.469493 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0892f-d8e7-4066-9c52-b42d82ac8ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:46.469769 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0892f-d8e7-4066-9c52-b42d82ac8ae1 None admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:46.470691 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec0892f-d8e7-4066-9c52-b42d82ac8ae1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:23:46.470998 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec0892f-d8e7-4066-9c52-b42d82ac8ae1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:46.472421 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec0892f-d8e7-4066-9c52-b42d82ac8ae1 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:46.490067 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 606/1227] 10.222.0.85 () {56 vars in 1143 bytes} [Tue Jun 7 14:23:46 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:23:46.496363 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1873c80-c649-49af-a8a8-9c5d6d0c48f7 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:46.596953 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1873c80-c649-49af-a8a8-9c5d6d0c48f7 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:46.631691 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc2e4e5-e56b-4107-b30f-30b4734b6128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:46.632502 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc2e4e5-e56b-4107-b30f-30b4734b6128 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:46.632787 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc2e4e5-e56b-4107-b30f-30b4734b6128 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:46.633064 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc2e4e5-e56b-4107-b30f-30b4734b6128 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:46.644322 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc2e4e5-e56b-4107-b30f-30b4734b6128 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:46.645796 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc2e4e5-e56b-4107-b30f-30b4734b6128 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:46.664066 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-6bc2e4e5-e56b-4107-b30f-30b4734b6128 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:23:46.664066 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:23:46.664066 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:23:46.664066 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:23:46.664066 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:23:46.664066 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:23:46.664066 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:23:46.664066 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:23:46.664066 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:23:46.664066 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:23:46.664066 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:23:46.664066 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:23:46.664066 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:23:46.664066 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:23:46.664066 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:23:46.665752 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:23:46.665752 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:23:46.665752 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:23:46.665752 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:23:46.665752 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:23:46.665752 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:23:46.665752 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:23:46.665752 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:23:46.665752 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:23:46.665752 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:23:46.665752 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:23:46.665752 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:23:46.665752 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:23:46.665752 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:23:46.665752 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:23:46.665752 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:23:46.665752 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:23:46.671437 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:23:46.671437 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:23:46.671437 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:23:46.671437 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:23:46.671437 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 622/1228] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:23:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:23:46.671851 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec0892f-d8e7-4066-9c52-b42d82ac8ae1 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:46.773860 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec0892f-d8e7-4066-9c52-b42d82ac8ae1 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:46.776195 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:46.808600 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbf5aab-c842-4d50-abfd-613054494efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:46.809260 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbf5aab-c842-4d50-abfd-613054494efa None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:46.809550 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbf5aab-c842-4d50-abfd-613054494efa None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:46.809827 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbf5aab-c842-4d50-abfd-613054494efa None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:46.810548 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbf5aab-c842-4d50-abfd-613054494efa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:23:46.810860 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbf5aab-c842-4d50-abfd-613054494efa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:46.812222 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbf5aab-c842-4d50-abfd-613054494efa None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:46.823348 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 607/1229] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jun 7 14:23:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:23:46.829274 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc2e4e5-e56b-4107-b30f-30b4734b6128 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:46.838650 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:46.928809 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc2e4e5-e56b-4107-b30f-30b4734b6128 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:46.961084 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2f4888-17a1-49cd-8547-f85b38213698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:46.962388 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2f4888-17a1-49cd-8547-f85b38213698 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:46.962672 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2f4888-17a1-49cd-8547-f85b38213698 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:46.962943 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2f4888-17a1-49cd-8547-f85b38213698 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:46.969782 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2f4888-17a1-49cd-8547-f85b38213698 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:46.971920 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2f4888-17a1-49cd-8547-f85b38213698 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:46.986332 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-3c2f4888-17a1-49cd-8547-f85b38213698 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:23:46.986332 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:23:46.986332 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:23:46.986332 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:23:46.986332 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:23:46.986332 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:23:46.986332 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:23:46.986332 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:23:46.986332 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:23:46.986332 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:23:46.986332 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:23:46.986332 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:23:46.986332 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 14:23:46.986332 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 14:23:46.986332 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 14:23:46.987407 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 14:23:46.987407 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:23:46.987407 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:23:46.987407 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:23:46.987407 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:23:46.987407 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:23:46.987407 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:23:46.987407 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:23:46.987407 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:23:46.987407 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:23:46.987407 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:23:46.987407 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:23:46.987407 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:23:46.987407 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:23:46.987407 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:23:46.987407 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:23:46.987407 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:23:46.988526 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 14:23:46.988526 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 14:23:46.988526 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 14:23:46.988526 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 14:23:46.988526 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 14:23:46.988526 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 14:23:46.988526 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:23:46.988526 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:23:46.988526 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 623/1230] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jun 7 14:23:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:23:46.993539 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbf5aab-c842-4d50-abfd-613054494efa None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:47.093358 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbf5aab-c842-4d50-abfd-613054494efa None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:47.131442 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110bce85-8730-4371-ba53-082616e636de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:47.132361 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110bce85-8730-4371-ba53-082616e636de None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:47.132361 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110bce85-8730-4371-ba53-082616e636de None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:47.132671 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110bce85-8730-4371-ba53-082616e636de None admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:47.133456 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110bce85-8730-4371-ba53-082616e636de None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:23:47.133797 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110bce85-8730-4371-ba53-082616e636de None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:47.135259 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110bce85-8730-4371-ba53-082616e636de None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:47.148356 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 608/1231] 10.222.0.85 () {56 vars in 1148 bytes} [Tue Jun 7 14:23:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:23:47.155077 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2f4888-17a1-49cd-8547-f85b38213698 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:47.219780 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:47.253587 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2f4888-17a1-49cd-8547-f85b38213698 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:47.286932 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87becd43-215f-4f6f-91e2-c901e42ea7b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:47.287816 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87becd43-215f-4f6f-91e2-c901e42ea7b2 None admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:47.288128 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87becd43-215f-4f6f-91e2-c901e42ea7b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:47.288416 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87becd43-215f-4f6f-91e2-c901e42ea7b2 None admin] PATH_INFO: `/v3/projects/724785eb74e44daf8736fd01fc0d1839/users/6330294b1c8e4dcd85aad189f007b0aa/roles/17516844e8b947b389d62f50718b852d` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:47.302555 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87becd43-215f-4f6f-91e2-c901e42ea7b2 None admin] RBAC: Authorizing `identity:create_grant(project_id=724785eb74e44daf8736fd01fc0d1839, user_id=6330294b1c8e4dcd85aad189f007b0aa, role_id=17516844e8b947b389d62f50718b852d)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:47.303983 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87becd43-215f-4f6f-91e2-c901e42ea7b2 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:47.388788 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 624/1232] 10.222.0.85 () {58 vars in 1439 bytes} [Tue Jun 7 14:23:47 2022] PUT /identity/v3/projects/724785eb74e44daf8736fd01fc0d1839/users/6330294b1c8e4dcd85aad189f007b0aa/roles/17516844e8b947b389d62f50718b852d => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:23:49.605756 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1743864-b4c8-47f2-9a75-32cc196b110c None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:49.606288 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1743864-b4c8-47f2-9a75-32cc196b110c None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:49.606288 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1743864-b4c8-47f2-9a75-32cc196b110c None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:49.607650 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 609/1233] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:23:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:23:49.612788 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d298ceb-214e-4a82-aa85-75692181f665 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:49.613210 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d298ceb-214e-4a82-aa85-75692181f665 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:49.613359 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d298ceb-214e-4a82-aa85-75692181f665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:49.630697 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-6d298ceb-214e-4a82-aa85-75692181f665 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:23:49.631872 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-6d298ceb-214e-4a82-aa85-75692181f665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:49.698374 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 625/1234] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:23:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:23:49.722229 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1743864-b4c8-47f2-9a75-32cc196b110c None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:49.747644 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:49.822939 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1743864-b4c8-47f2-9a75-32cc196b110c None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:49.857871 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103099a6-6610-4b3d-aab7-92316339b45e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:49.858585 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103099a6-6610-4b3d-aab7-92316339b45e None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:49.858871 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103099a6-6610-4b3d-aab7-92316339b45e None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:49.859152 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103099a6-6610-4b3d-aab7-92316339b45e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:49.870108 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103099a6-6610-4b3d-aab7-92316339b45e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:23:49.878128 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-103099a6-6610-4b3d-aab7-92316339b45e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:23:49.878128 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:23:49.878128 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:23:49.878128 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:23:49.878128 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:23:49.878128 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:23:49.878128 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:23:49.878128 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:23:49.878128 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:23:49.878128 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:23:49.878128 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:23:49.878128 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:23:49.878128 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:23:49.878128 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:23:49.878128 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:23:49.878128 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:23:49.879310 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:23:49.879310 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:23:49.879310 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:23:49.879310 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:23:49.879310 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:23:49.879310 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:23:49.879310 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:23:49.879310 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:23:49.879310 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:23:49.879310 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:23:49.879310 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:23:49.879310 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:23:49.879310 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:23:49.879310 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:23:49.879310 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:23:49.879310 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:23:49.880529 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:23:49.880529 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:23:49.880529 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:23:49.880529 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:23:49.880529 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:23:49.880529 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:23:49.880529 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:23:49.880529 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:23:49.880529 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:23:49.880529 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:23:49.880529 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 14:23:49.880529 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:23:49.880529 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 610/1235] 10.222.0.85 () {56 vars in 1102 bytes} [Tue Jun 7 14:23:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:23:49.885895 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d298ceb-214e-4a82-aa85-75692181f665 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:49.931742 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:49.985404 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d298ceb-214e-4a82-aa85-75692181f665 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:50.019196 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb579bf2-bcde-48d2-b2db-cfae5db283bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:50.019861 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb579bf2-bcde-48d2-b2db-cfae5db283bb None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:50.020324 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb579bf2-bcde-48d2-b2db-cfae5db283bb None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:50.020637 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb579bf2-bcde-48d2-b2db-cfae5db283bb None admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:50.021409 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb579bf2-bcde-48d2-b2db-cfae5db283bb None admin] RBAC: Adding query filter params (name=service) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:23:50.025978 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb579bf2-bcde-48d2-b2db-cfae5db283bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:50.025978 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb579bf2-bcde-48d2-b2db-cfae5db283bb None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:50.030744 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 626/1236] 10.222.0.85 () {56 vars in 1103 bytes} [Tue Jun 7 14:23:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:23:50.036634 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103099a6-6610-4b3d-aab7-92316339b45e None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:50.121475 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:50.136992 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103099a6-6610-4b3d-aab7-92316339b45e None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:50.170117 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe55834-2fc8-424e-ac0f-a5be9b1fb885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:50.170904 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe55834-2fc8-424e-ac0f-a5be9b1fb885 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:50.171195 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe55834-2fc8-424e-ac0f-a5be9b1fb885 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:50.171476 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe55834-2fc8-424e-ac0f-a5be9b1fb885 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:50.181824 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe55834-2fc8-424e-ac0f-a5be9b1fb885 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:50.183285 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe55834-2fc8-424e-ac0f-a5be9b1fb885 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:50.200028 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-1fe55834-2fc8-424e-ac0f-a5be9b1fb885 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:23:50.200028 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:23:50.200028 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:23:50.200028 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:23:50.200028 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:23:50.200028 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:23:50.200028 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:23:50.200028 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:23:50.200028 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:23:50.200028 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:23:50.200028 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:23:50.200028 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:23:50.200028 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:23:50.200028 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:23:50.200028 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:23:50.201170 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:23:50.201170 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:23:50.201170 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:23:50.201170 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:23:50.201170 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:23:50.201170 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:23:50.201170 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:23:50.201170 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:23:50.201170 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:23:50.201170 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:23:50.201170 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:23:50.201170 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:23:50.201170 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:23:50.201170 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:23:50.201170 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:23:50.201170 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:23:50.201170 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:23:50.202192 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:23:50.202192 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:23:50.202192 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:23:50.202192 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:23:50.202192 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 611/1237] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:23:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:23:50.207589 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb579bf2-bcde-48d2-b2db-cfae5db283bb None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:50.316060 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb579bf2-bcde-48d2-b2db-cfae5db283bb None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:50.331271 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:50.355101 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1147608e-f411-42dd-9ffc-3a3dfd53263a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:50.355759 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1147608e-f411-42dd-9ffc-3a3dfd53263a None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:50.356039 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1147608e-f411-42dd-9ffc-3a3dfd53263a None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:50.356313 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1147608e-f411-42dd-9ffc-3a3dfd53263a None admin] PATH_INFO: `/v3/domains` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:50.357063 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1147608e-f411-42dd-9ffc-3a3dfd53263a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:23:50.357386 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1147608e-f411-42dd-9ffc-3a3dfd53263a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:50.358745 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1147608e-f411-42dd-9ffc-3a3dfd53263a None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:50.370442 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 627/1238] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jun 7 14:23:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:23:50.380270 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe55834-2fc8-424e-ac0f-a5be9b1fb885 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:50.476014 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe55834-2fc8-424e-ac0f-a5be9b1fb885 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:50.514442 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d7b79c-0160-4952-9bbd-413e705097bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:50.515921 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d7b79c-0160-4952-9bbd-413e705097bc None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:50.516209 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d7b79c-0160-4952-9bbd-413e705097bc None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:50.516521 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d7b79c-0160-4952-9bbd-413e705097bc None admin] PATH_INFO: `/v3/users/placement` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:50.522340 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:50.528357 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d7b79c-0160-4952-9bbd-413e705097bc None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:50.529935 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d7b79c-0160-4952-9bbd-413e705097bc None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:50.544603 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-85d7b79c-0160-4952-9bbd-413e705097bc None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 07 14:23:50.544603 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:23:50.544603 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:23:50.544603 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:23:50.544603 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:23:50.544603 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:23:50.544603 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:23:50.544603 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:23:50.544603 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:23:50.544603 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:23:50.544603 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:23:50.544603 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:23:50.544603 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 14:23:50.544603 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 14:23:50.544603 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 14:23:50.544603 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 14:23:50.545788 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:23:50.545788 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:23:50.545788 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 14:23:50.545788 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:23:50.545788 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 14:23:50.545788 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:23:50.545788 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:23:50.545788 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:23:50.545788 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:23:50.545788 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:23:50.545788 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:23:50.545788 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:23:50.545788 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:23:50.545788 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:23:50.545788 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:23:50.545788 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:23:50.545788 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:23:50.545788 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:23:50.546928 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:23:50.546928 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:23:50.546928 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 14:23:50.546928 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 14:23:50.546928 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 14:23:50.546928 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 14:23:50.546928 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 14:23:50.546928 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 14:23:50.546928 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 07 14:23:50.546928 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:23:50.548521 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 612/1239] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:23:50 2022] GET /identity/v3/users/placement => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:23:50.552697 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1147608e-f411-42dd-9ffc-3a3dfd53263a None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:50.652402 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1147608e-f411-42dd-9ffc-3a3dfd53263a None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:50.686451 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d571e7-126e-4498-8cf8-2ed88c1d48f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:50.687179 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d571e7-126e-4498-8cf8-2ed88c1d48f7 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:50.687460 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d571e7-126e-4498-8cf8-2ed88c1d48f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:50.687729 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d571e7-126e-4498-8cf8-2ed88c1d48f7 None admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:50.688599 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d571e7-126e-4498-8cf8-2ed88c1d48f7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:23:50.688904 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d571e7-126e-4498-8cf8-2ed88c1d48f7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:50.690333 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d571e7-126e-4498-8cf8-2ed88c1d48f7 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:50.709917 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 628/1240] 10.222.0.85 () {56 vars in 1143 bytes} [Tue Jun 7 14:23:50 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:23:50.716104 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d7b79c-0160-4952-9bbd-413e705097bc None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:50.820931 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d7b79c-0160-4952-9bbd-413e705097bc None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:50.861838 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eaa2b7-e2d5-4a23-b69b-06cca1676ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:50.862489 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eaa2b7-e2d5-4a23-b69b-06cca1676ed9 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:50.862776 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eaa2b7-e2d5-4a23-b69b-06cca1676ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:50.863053 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eaa2b7-e2d5-4a23-b69b-06cca1676ed9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:50.923802 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:50.930356 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eaa2b7-e2d5-4a23-b69b-06cca1676ed9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:50.931796 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eaa2b7-e2d5-4a23-b69b-06cca1676ed9 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:51.029475 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 613/1241] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jun 7 14:23:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 14:23:51.035336 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d571e7-126e-4498-8cf8-2ed88c1d48f7 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:51.062379 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:51.136775 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d571e7-126e-4498-8cf8-2ed88c1d48f7 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:51.178997 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebece1e3-3638-4ec0-8a4a-d07e8186334d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:51.179735 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebece1e3-3638-4ec0-8a4a-d07e8186334d None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:51.180024 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebece1e3-3638-4ec0-8a4a-d07e8186334d None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:51.180363 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebece1e3-3638-4ec0-8a4a-d07e8186334d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:51.190559 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebece1e3-3638-4ec0-8a4a-d07e8186334d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:51.192041 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebece1e3-3638-4ec0-8a4a-d07e8186334d None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:51.210085 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-ebece1e3-3638-4ec0-8a4a-d07e8186334d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:23:51.210085 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:23:51.210085 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:23:51.210085 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:23:51.210085 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:23:51.210085 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:23:51.210085 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:23:51.210085 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:23:51.210085 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:23:51.210085 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:23:51.210085 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:23:51.210085 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:23:51.210085 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:23:51.210085 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:23:51.210085 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:23:51.211172 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:23:51.211172 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:23:51.211172 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:23:51.211172 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:23:51.211172 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:23:51.211172 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:23:51.211172 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:23:51.211172 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:23:51.211172 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:23:51.211172 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:23:51.211172 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:23:51.211172 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:23:51.211172 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:23:51.211172 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:23:51.211172 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:23:51.211172 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:23:51.211172 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:23:51.212895 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:23:51.212895 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:23:51.212895 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:23:51.212895 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:23:51.212895 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 629/1242] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:23:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:23:51.217636 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eaa2b7-e2d5-4a23-b69b-06cca1676ed9 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:51.309979 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:51.315706 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eaa2b7-e2d5-4a23-b69b-06cca1676ed9 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:51.349375 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82cc189-e1ae-4974-bae9-4b803b6b1ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:51.350039 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82cc189-e1ae-4974-bae9-4b803b6b1ff6 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:51.350316 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82cc189-e1ae-4974-bae9-4b803b6b1ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:51.350591 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82cc189-e1ae-4974-bae9-4b803b6b1ff6 None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:51.351302 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82cc189-e1ae-4974-bae9-4b803b6b1ff6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:23:51.351610 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82cc189-e1ae-4974-bae9-4b803b6b1ff6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:51.352983 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82cc189-e1ae-4974-bae9-4b803b6b1ff6 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:51.363697 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 614/1243] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jun 7 14:23:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:23:51.369958 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebece1e3-3638-4ec0-8a4a-d07e8186334d None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:51.451351 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:51.473058 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebece1e3-3638-4ec0-8a4a-d07e8186334d None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:51.508810 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f43ee15-b948-4b82-a566-76288e80272f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:51.509608 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f43ee15-b948-4b82-a566-76288e80272f None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:51.509927 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f43ee15-b948-4b82-a566-76288e80272f None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:51.510206 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f43ee15-b948-4b82-a566-76288e80272f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:51.517477 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f43ee15-b948-4b82-a566-76288e80272f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:51.519063 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f43ee15-b948-4b82-a566-76288e80272f None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:51.533739 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-3f43ee15-b948-4b82-a566-76288e80272f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:23:51.533739 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:23:51.533739 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:23:51.533739 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:23:51.533739 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:23:51.533739 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:23:51.533739 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:23:51.533739 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:23:51.533739 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:23:51.533739 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:23:51.533739 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:23:51.533739 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:23:51.533739 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 14:23:51.533739 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 14:23:51.533739 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 14:23:51.534830 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 14:23:51.534830 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:23:51.534830 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:23:51.534830 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:23:51.534830 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:23:51.534830 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:23:51.534830 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:23:51.534830 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:23:51.534830 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:23:51.534830 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:23:51.534830 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:23:51.534830 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:23:51.534830 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:23:51.534830 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:23:51.534830 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:23:51.534830 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:23:51.534830 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:23:51.535885 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 14:23:51.535885 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 14:23:51.535885 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 14:23:51.535885 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 14:23:51.535885 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 14:23:51.535885 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 14:23:51.535885 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:23:51.535885 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:23:51.535885 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 630/1244] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jun 7 14:23:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:23:51.541594 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82cc189-e1ae-4974-bae9-4b803b6b1ff6 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:51.645257 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82cc189-e1ae-4974-bae9-4b803b6b1ff6 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:51.679248 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45952400-c5b6-4abf-84b5-450a01b432ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:51.679907 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45952400-c5b6-4abf-84b5-450a01b432ab None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:51.680194 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45952400-c5b6-4abf-84b5-450a01b432ab None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:51.680527 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45952400-c5b6-4abf-84b5-450a01b432ab None admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:51.681318 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45952400-c5b6-4abf-84b5-450a01b432ab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:23:51.681629 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45952400-c5b6-4abf-84b5-450a01b432ab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:51.683087 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45952400-c5b6-4abf-84b5-450a01b432ab None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:51.694809 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 615/1245] 10.222.0.85 () {56 vars in 1148 bytes} [Tue Jun 7 14:23:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:23:51.700992 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f43ee15-b948-4b82-a566-76288e80272f None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:51.803648 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f43ee15-b948-4b82-a566-76288e80272f None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:51.838737 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c3532b-558e-4b2a-88ad-345a6a41b9f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:51.839401 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c3532b-558e-4b2a-88ad-345a6a41b9f7 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:51.839690 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c3532b-558e-4b2a-88ad-345a6a41b9f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:51.839971 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c3532b-558e-4b2a-88ad-345a6a41b9f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:51.840843 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c3532b-558e-4b2a-88ad-345a6a41b9f7 None admin] RBAC: Adding query filter params (role.id=17516844e8b947b389d62f50718b852d, scope.project.id=724785eb74e44daf8736fd01fc0d1839, user.id=6330294b1c8e4dcd85aad189f007b0aa) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:23:51.841162 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c3532b-558e-4b2a-88ad-345a6a41b9f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:51.842592 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c3532b-558e-4b2a-88ad-345a6a41b9f7 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:51.848728 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 631/1246] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jun 7 14:23:51 2022] GET /identity/v3/role_assignments?user.id=6330294b1c8e4dcd85aad189f007b0aa&scope.project.id=724785eb74e44daf8736fd01fc0d1839&role.id=17516844e8b947b389d62f50718b852d => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:23:54.180244 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313f8829-57d6-4c1f-828d-444b0d416b9a None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:54.180877 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313f8829-57d6-4c1f-828d-444b0d416b9a None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:54.180877 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313f8829-57d6-4c1f-828d-444b0d416b9a None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:54.182221 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 616/1247] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:23:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:23:54.187642 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5cfbf6-0d23-4860-a10c-0cade5f9bea9 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:54.193786 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5cfbf6-0d23-4860-a10c-0cade5f9bea9 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:54.193786 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5cfbf6-0d23-4860-a10c-0cade5f9bea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:54.193786 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:54.206608 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-ec5cfbf6-0d23-4860-a10c-0cade5f9bea9 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:23:54.207796 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-ec5cfbf6-0d23-4860-a10c-0cade5f9bea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:54.276624 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 632/1248] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:23:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:23:54.300421 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313f8829-57d6-4c1f-828d-444b0d416b9a None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:54.318499 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:54.404950 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313f8829-57d6-4c1f-828d-444b0d416b9a None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:54.440902 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d1908c-987e-4666-894a-a8aa94741e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:54.441708 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d1908c-987e-4666-894a-a8aa94741e46 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:54.441962 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d1908c-987e-4666-894a-a8aa94741e46 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:54.442243 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d1908c-987e-4666-894a-a8aa94741e46 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:54.452800 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d1908c-987e-4666-894a-a8aa94741e46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:23:54.460236 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-d0d1908c-987e-4666-894a-a8aa94741e46 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:23:54.460236 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:23:54.460236 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:23:54.460236 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:23:54.460236 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:23:54.460236 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:23:54.460236 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:23:54.460236 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:23:54.460236 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:23:54.460236 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:23:54.460236 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:23:54.460236 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:23:54.460236 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:23:54.460236 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:23:54.460236 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:23:54.460236 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:23:54.461448 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:23:54.461448 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:23:54.461448 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:23:54.461448 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:23:54.461448 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:23:54.461448 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:23:54.461448 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:23:54.461448 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:23:54.461448 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:23:54.461448 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:23:54.461448 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:23:54.461448 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:23:54.461448 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:23:54.461448 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:23:54.461448 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:23:54.461448 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:23:54.462522 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:23:54.462522 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:23:54.462522 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:23:54.462522 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:23:54.462522 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:23:54.462522 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:23:54.462522 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:23:54.462522 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:23:54.462522 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:23:54.462522 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:23:54.462522 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:23:54.462522 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:23:54.462522 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 617/1249] 10.222.0.85 () {56 vars in 1096 bytes} [Tue Jun 7 14:23:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:23:54.467484 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5cfbf6-0d23-4860-a10c-0cade5f9bea9 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:54.566934 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5cfbf6-0d23-4860-a10c-0cade5f9bea9 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:54.596301 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:54.601225 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8acd8d-8c22-4d54-bd3c-949a5aa9e4e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:54.602850 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8acd8d-8c22-4d54-bd3c-949a5aa9e4e7 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:54.602850 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8acd8d-8c22-4d54-bd3c-949a5aa9e4e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:54.602850 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8acd8d-8c22-4d54-bd3c-949a5aa9e4e7 None admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:54.603265 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8acd8d-8c22-4d54-bd3c-949a5aa9e4e7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:23:54.603572 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8acd8d-8c22-4d54-bd3c-949a5aa9e4e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:54.604987 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8acd8d-8c22-4d54-bd3c-949a5aa9e4e7 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:54.612988 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 633/1250] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jun 7 14:23:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:23:54.619298 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d1908c-987e-4666-894a-a8aa94741e46 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:54.698883 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:54.723827 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d1908c-987e-4666-894a-a8aa94741e46 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:54.761409 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132707b4-e575-4fef-84f1-3b921c9db737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:54.762211 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132707b4-e575-4fef-84f1-3b921c9db737 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:54.762497 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132707b4-e575-4fef-84f1-3b921c9db737 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:54.762772 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132707b4-e575-4fef-84f1-3b921c9db737 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:54.773464 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132707b4-e575-4fef-84f1-3b921c9db737 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:54.774977 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132707b4-e575-4fef-84f1-3b921c9db737 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:54.792514 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-132707b4-e575-4fef-84f1-3b921c9db737 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:23:54.792514 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:23:54.792514 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:23:54.792514 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:23:54.792514 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:23:54.792514 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:23:54.792514 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:23:54.792514 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:23:54.792514 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:23:54.792514 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:23:54.792514 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:23:54.792514 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:23:54.792514 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:23:54.792514 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:23:54.792514 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:23:54.793605 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:23:54.793605 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:23:54.793605 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:23:54.793605 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:23:54.793605 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:23:54.793605 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:23:54.793605 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:23:54.793605 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:23:54.793605 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:23:54.793605 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:23:54.793605 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:23:54.793605 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:23:54.793605 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:23:54.793605 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:23:54.793605 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:23:54.793605 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:23:54.793605 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:23:54.794871 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:23:54.794871 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:23:54.794871 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:23:54.794871 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:23:54.794871 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 618/1251] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:23:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:23:54.799409 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8acd8d-8c22-4d54-bd3c-949a5aa9e4e7 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:54.897983 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8acd8d-8c22-4d54-bd3c-949a5aa9e4e7 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:54.931537 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bb7126-4160-4cd6-a861-ed490d2752e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:54.932204 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bb7126-4160-4cd6-a861-ed490d2752e7 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:54.932520 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bb7126-4160-4cd6-a861-ed490d2752e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:54.932800 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bb7126-4160-4cd6-a861-ed490d2752e7 None admin] PATH_INFO: `/v3/domains` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:54.933535 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bb7126-4160-4cd6-a861-ed490d2752e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:23:54.933827 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bb7126-4160-4cd6-a861-ed490d2752e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:54.935212 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bb7126-4160-4cd6-a861-ed490d2752e7 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:54.947174 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 634/1252] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jun 7 14:23:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:23:54.953007 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132707b4-e575-4fef-84f1-3b921c9db737 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:55.053543 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132707b4-e575-4fef-84f1-3b921c9db737 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:55.079595 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:55.087880 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782b53f4-67d4-413e-93ee-3ad4e2925342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:55.088718 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b53f4-67d4-413e-93ee-3ad4e2925342 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:55.089003 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b53f4-67d4-413e-93ee-3ad4e2925342 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:55.089278 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b53f4-67d4-413e-93ee-3ad4e2925342 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:55.096599 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782b53f4-67d4-413e-93ee-3ad4e2925342 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:55.098032 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782b53f4-67d4-413e-93ee-3ad4e2925342 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:55.112545 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-782b53f4-67d4-413e-93ee-3ad4e2925342 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 07 14:23:55.112545 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:23:55.112545 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:23:55.112545 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:23:55.112545 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:23:55.112545 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:23:55.112545 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:23:55.112545 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:23:55.112545 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:23:55.112545 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:23:55.112545 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:23:55.112545 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:23:55.112545 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 14:23:55.112545 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 14:23:55.112545 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 14:23:55.112545 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 14:23:55.113674 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:23:55.113674 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:23:55.113674 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 14:23:55.113674 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:23:55.113674 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 14:23:55.113674 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:23:55.113674 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:23:55.113674 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:23:55.113674 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:23:55.113674 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:23:55.113674 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:23:55.113674 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:23:55.113674 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:23:55.113674 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:23:55.113674 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:23:55.113674 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:23:55.113674 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:23:55.113674 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:23:55.114930 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:23:55.114930 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:23:55.114930 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 14:23:55.114930 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 14:23:55.114930 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 14:23:55.114930 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 14:23:55.114930 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 14:23:55.114930 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 14:23:55.114930 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 07 14:23:55.114930 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:23:55.114930 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 619/1253] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:23:54 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:23:55.119662 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bb7126-4160-4cd6-a861-ed490d2752e7 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:55.175425 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:55.221495 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bb7126-4160-4cd6-a861-ed490d2752e7 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:55.254507 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a78573-96ed-4b01-96d1-bca053edaecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:55.255220 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a78573-96ed-4b01-96d1-bca053edaecd None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:55.255512 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a78573-96ed-4b01-96d1-bca053edaecd None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:55.255794 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a78573-96ed-4b01-96d1-bca053edaecd None admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:55.256661 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a78573-96ed-4b01-96d1-bca053edaecd None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:23:55.256976 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a78573-96ed-4b01-96d1-bca053edaecd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:55.258399 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a78573-96ed-4b01-96d1-bca053edaecd None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:55.276464 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 635/1254] 10.222.0.85 () {56 vars in 1143 bytes} [Tue Jun 7 14:23:55 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:23:55.289013 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782b53f4-67d4-413e-93ee-3ad4e2925342 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:55.381680 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782b53f4-67d4-413e-93ee-3ad4e2925342 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:55.414989 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8f6b19-3df2-4657-8cf2-870e9bd8d501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:55.416187 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8f6b19-3df2-4657-8cf2-870e9bd8d501 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:55.416501 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8f6b19-3df2-4657-8cf2-870e9bd8d501 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:55.416784 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8f6b19-3df2-4657-8cf2-870e9bd8d501 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:55.470683 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:55.482600 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8f6b19-3df2-4657-8cf2-870e9bd8d501 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:55.484048 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8f6b19-3df2-4657-8cf2-870e9bd8d501 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:55.581986 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 620/1255] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jun 7 14:23:55 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 14:23:55.588695 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a78573-96ed-4b01-96d1-bca053edaecd None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:55.686775 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a78573-96ed-4b01-96d1-bca053edaecd None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:55.701821 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:55.721996 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6651b98e-38da-445d-a9d0-81879bfb4c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:55.722717 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6651b98e-38da-445d-a9d0-81879bfb4c0c None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:55.722998 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6651b98e-38da-445d-a9d0-81879bfb4c0c None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:55.723272 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6651b98e-38da-445d-a9d0-81879bfb4c0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:55.732887 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6651b98e-38da-445d-a9d0-81879bfb4c0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:55.735352 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6651b98e-38da-445d-a9d0-81879bfb4c0c None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:55.752751 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-6651b98e-38da-445d-a9d0-81879bfb4c0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:23:55.752751 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:23:55.752751 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:23:55.752751 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:23:55.752751 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:23:55.752751 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:23:55.752751 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:23:55.752751 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:23:55.752751 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:23:55.752751 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:23:55.752751 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:23:55.752751 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:23:55.752751 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:23:55.752751 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:23:55.752751 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:23:55.753832 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:23:55.753832 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:23:55.753832 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:23:55.753832 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:23:55.753832 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:23:55.753832 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:23:55.753832 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:23:55.753832 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:23:55.753832 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:23:55.753832 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:23:55.753832 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:23:55.753832 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:23:55.753832 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:23:55.753832 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:23:55.753832 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:23:55.753832 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:23:55.753832 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:23:55.754956 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:23:55.754956 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:23:55.754956 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:23:55.754956 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:23:55.754956 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 636/1256] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:23:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:23:55.760115 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8f6b19-3df2-4657-8cf2-870e9bd8d501 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:55.847754 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:55.859208 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8f6b19-3df2-4657-8cf2-870e9bd8d501 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:55.892811 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5e8e7b-75a6-4793-a4c4-886b9d9159dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:55.893470 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5e8e7b-75a6-4793-a4c4-886b9d9159dc None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:55.893749 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5e8e7b-75a6-4793-a4c4-886b9d9159dc None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:55.894020 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5e8e7b-75a6-4793-a4c4-886b9d9159dc None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:55.894741 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5e8e7b-75a6-4793-a4c4-886b9d9159dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:23:55.895046 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5e8e7b-75a6-4793-a4c4-886b9d9159dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:55.896424 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5e8e7b-75a6-4793-a4c4-886b9d9159dc None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:55.908109 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 621/1257] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jun 7 14:23:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:23:55.913941 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6651b98e-38da-445d-a9d0-81879bfb4c0c None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:56.015613 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6651b98e-38da-445d-a9d0-81879bfb4c0c None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:56.051559 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad789bf-a15e-4a4a-b440-93e3b465c41f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:56.052359 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad789bf-a15e-4a4a-b440-93e3b465c41f None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:56.052674 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad789bf-a15e-4a4a-b440-93e3b465c41f None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:56.052953 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad789bf-a15e-4a4a-b440-93e3b465c41f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:56.060139 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad789bf-a15e-4a4a-b440-93e3b465c41f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:56.061668 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad789bf-a15e-4a4a-b440-93e3b465c41f None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:56.075177 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-8ad789bf-a15e-4a4a-b440-93e3b465c41f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:23:56.075177 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:23:56.075177 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:23:56.075177 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:23:56.075177 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:23:56.075177 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:23:56.075177 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:23:56.075177 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:23:56.075177 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:23:56.075177 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:23:56.075177 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:23:56.075177 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:23:56.075177 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 14:23:56.075177 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 14:23:56.075177 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 14:23:56.076255 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 14:23:56.076255 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:23:56.076255 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:23:56.076255 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:23:56.076255 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:23:56.076255 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:23:56.076255 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:23:56.076255 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:23:56.076255 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:23:56.076255 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:23:56.076255 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:23:56.076255 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:23:56.076255 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:23:56.076255 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:23:56.076255 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:23:56.076255 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:23:56.076255 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:23:56.077213 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 14:23:56.077213 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 14:23:56.077213 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 14:23:56.077213 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 14:23:56.077213 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 14:23:56.077213 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 14:23:56.077213 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:23:56.077213 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:23:56.077213 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 637/1258] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jun 7 14:23:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:23:56.082149 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5e8e7b-75a6-4793-a4c4-886b9d9159dc None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:56.182781 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5e8e7b-75a6-4793-a4c4-886b9d9159dc None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:56.217000 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d258c92b-5e34-4110-8f57-4397b85699cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:56.217659 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d258c92b-5e34-4110-8f57-4397b85699cf None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:56.217944 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d258c92b-5e34-4110-8f57-4397b85699cf None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:56.218221 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d258c92b-5e34-4110-8f57-4397b85699cf None admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:56.219005 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d258c92b-5e34-4110-8f57-4397b85699cf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:23:56.219317 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d258c92b-5e34-4110-8f57-4397b85699cf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:56.220864 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d258c92b-5e34-4110-8f57-4397b85699cf None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:56.233412 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 622/1259] 10.222.0.85 () {56 vars in 1148 bytes} [Tue Jun 7 14:23:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:23:56.239533 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad789bf-a15e-4a4a-b440-93e3b465c41f None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:56.265602 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:56.338029 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad789bf-a15e-4a4a-b440-93e3b465c41f None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:56.373847 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6245eb35-fce4-482a-85b2-cab1adaa4a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:56.374597 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6245eb35-fce4-482a-85b2-cab1adaa4a7c None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:56.374877 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6245eb35-fce4-482a-85b2-cab1adaa4a7c None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:56.375151 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6245eb35-fce4-482a-85b2-cab1adaa4a7c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:56.376036 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6245eb35-fce4-482a-85b2-cab1adaa4a7c None admin] RBAC: Adding query filter params (role.id=9baff2b18f2649f7b757ff2e1f63c853, scope.project.id=724785eb74e44daf8736fd01fc0d1839, user.id=6330294b1c8e4dcd85aad189f007b0aa) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:23:56.376350 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6245eb35-fce4-482a-85b2-cab1adaa4a7c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:56.377879 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6245eb35-fce4-482a-85b2-cab1adaa4a7c None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:56.383337 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 638/1260] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jun 7 14:23:56 2022] GET /identity/v3/role_assignments?user.id=6330294b1c8e4dcd85aad189f007b0aa&scope.project.id=724785eb74e44daf8736fd01fc0d1839&role.id=9baff2b18f2649f7b757ff2e1f63c853 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:23:58.626506 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407ad53f-d3bc-4f4e-afcf-d6346a827781 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:58.627097 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407ad53f-d3bc-4f4e-afcf-d6346a827781 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:58.627097 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407ad53f-d3bc-4f4e-afcf-d6346a827781 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:58.628452 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 623/1261] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:23:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:23:58.633722 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5118a90b-9bb2-4d7b-b851-2825e8e6bec3 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:58.634641 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5118a90b-9bb2-4d7b-b851-2825e8e6bec3 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:58.634814 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5118a90b-9bb2-4d7b-b851-2825e8e6bec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:58.652246 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-5118a90b-9bb2-4d7b-b851-2825e8e6bec3 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:23:58.653447 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-5118a90b-9bb2-4d7b-b851-2825e8e6bec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:23:58.721129 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 639/1262] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:23:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:23:58.744682 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407ad53f-d3bc-4f4e-afcf-d6346a827781 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:58.755884 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:58.846885 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407ad53f-d3bc-4f4e-afcf-d6346a827781 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:58.884714 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdc0fe9-8cf3-4421-9195-d0a7f5789247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:58.884714 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdc0fe9-8cf3-4421-9195-d0a7f5789247 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:58.884714 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdc0fe9-8cf3-4421-9195-d0a7f5789247 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:58.884714 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdc0fe9-8cf3-4421-9195-d0a7f5789247 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:58.894779 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdc0fe9-8cf3-4421-9195-d0a7f5789247 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:23:58.903037 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-6cdc0fe9-8cf3-4421-9195-d0a7f5789247 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:23:58.903037 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:23:58.903037 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:23:58.903037 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:23:58.903037 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:23:58.903037 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:23:58.903037 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:23:58.903037 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:23:58.903037 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:23:58.903037 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:23:58.903037 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:23:58.903037 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:23:58.903037 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:23:58.903037 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:23:58.903037 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:23:58.903037 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:23:58.906482 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:23:58.906482 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:23:58.906482 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:23:58.906482 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:23:58.906482 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:23:58.906482 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:23:58.906482 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:23:58.906482 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:23:58.906482 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:23:58.906482 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:23:58.906482 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:23:58.906482 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:23:58.906482 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:23:58.906482 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:23:58.906482 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:23:58.906482 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:23:58.907534 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:23:58.907534 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:23:58.907534 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:23:58.907534 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:23:58.907534 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:23:58.907534 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:23:58.907534 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:23:58.907534 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:23:58.907534 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:23:58.907534 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:23:58.907534 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:23:58.907534 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:23:58.907534 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 624/1263] 10.222.0.85 () {56 vars in 1096 bytes} [Tue Jun 7 14:23:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 14:23:58.910396 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5118a90b-9bb2-4d7b-b851-2825e8e6bec3 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:58.929797 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:59.012206 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5118a90b-9bb2-4d7b-b851-2825e8e6bec3 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:59.046907 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc4a6c8-facd-4781-bc2f-0ed49bed0f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:59.047572 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc4a6c8-facd-4781-bc2f-0ed49bed0f37 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:59.047859 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc4a6c8-facd-4781-bc2f-0ed49bed0f37 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:59.048137 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc4a6c8-facd-4781-bc2f-0ed49bed0f37 None admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:59.048923 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc4a6c8-facd-4781-bc2f-0ed49bed0f37 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:23:59.053886 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc4a6c8-facd-4781-bc2f-0ed49bed0f37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:59.053886 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc4a6c8-facd-4781-bc2f-0ed49bed0f37 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:59.058499 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 640/1264] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jun 7 14:23:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:23:59.064491 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdc0fe9-8cf3-4421-9195-d0a7f5789247 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:59.130404 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:59.162574 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdc0fe9-8cf3-4421-9195-d0a7f5789247 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:59.195127 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1c20da-523d-4b0d-8f20-fe46e748c40a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:59.195910 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1c20da-523d-4b0d-8f20-fe46e748c40a None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:59.196193 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1c20da-523d-4b0d-8f20-fe46e748c40a None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:59.197051 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1c20da-523d-4b0d-8f20-fe46e748c40a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:59.207496 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1c20da-523d-4b0d-8f20-fe46e748c40a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:59.208987 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1c20da-523d-4b0d-8f20-fe46e748c40a None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:59.227302 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-5a1c20da-523d-4b0d-8f20-fe46e748c40a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:23:59.227302 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:23:59.227302 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:23:59.227302 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:23:59.227302 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:23:59.227302 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:23:59.227302 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:23:59.227302 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:23:59.227302 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:23:59.227302 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:23:59.227302 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:23:59.227302 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:23:59.227302 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:23:59.227302 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:23:59.227302 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:23:59.232677 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:23:59.232677 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:23:59.232677 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:23:59.232677 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:23:59.232677 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:23:59.232677 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:23:59.232677 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:23:59.232677 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:23:59.232677 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:23:59.232677 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:23:59.232677 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:23:59.232677 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:23:59.232677 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:23:59.232677 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:23:59.232677 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:23:59.232677 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:23:59.232677 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:23:59.233744 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:23:59.233744 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:23:59.233744 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:23:59.233744 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:23:59.233744 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 625/1265] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:23:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:23:59.234457 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc4a6c8-facd-4781-bc2f-0ed49bed0f37 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:59.316796 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:59.335524 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc4a6c8-facd-4781-bc2f-0ed49bed0f37 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:59.368278 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324de8aa-d360-4834-b3f7-9b0a9a3439aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:59.369543 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324de8aa-d360-4834-b3f7-9b0a9a3439aa None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:59.369829 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324de8aa-d360-4834-b3f7-9b0a9a3439aa None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:59.370103 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324de8aa-d360-4834-b3f7-9b0a9a3439aa None admin] PATH_INFO: `/v3/domains` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:59.370823 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324de8aa-d360-4834-b3f7-9b0a9a3439aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:23:59.371136 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324de8aa-d360-4834-b3f7-9b0a9a3439aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:59.372793 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324de8aa-d360-4834-b3f7-9b0a9a3439aa None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:59.384015 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 641/1266] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jun 7 14:23:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:23:59.389649 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1c20da-523d-4b0d-8f20-fe46e748c40a None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:59.488112 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1c20da-523d-4b0d-8f20-fe46e748c40a None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:59.508194 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:59.522550 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bb9760-0ed6-4075-bae9-2ebb3ee85b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:59.523361 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bb9760-0ed6-4075-bae9-2ebb3ee85b98 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:59.523651 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bb9760-0ed6-4075-bae9-2ebb3ee85b98 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:59.523939 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bb9760-0ed6-4075-bae9-2ebb3ee85b98 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:59.531464 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bb9760-0ed6-4075-bae9-2ebb3ee85b98 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:59.532949 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bb9760-0ed6-4075-bae9-2ebb3ee85b98 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:59.547458 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-64bb9760-0ed6-4075-bae9-2ebb3ee85b98 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 07 14:23:59.547458 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:23:59.547458 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:23:59.547458 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:23:59.547458 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:23:59.547458 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:23:59.547458 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:23:59.547458 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:23:59.547458 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:23:59.547458 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:23:59.547458 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:23:59.547458 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:23:59.547458 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 14:23:59.547458 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 14:23:59.547458 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 14:23:59.547458 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 14:23:59.548651 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:23:59.548651 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:23:59.548651 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 14:23:59.548651 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:23:59.548651 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 14:23:59.548651 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:23:59.548651 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:23:59.548651 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:23:59.548651 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:23:59.548651 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:23:59.548651 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:23:59.548651 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:23:59.548651 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:23:59.548651 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:23:59.548651 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:23:59.548651 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:23:59.548651 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:23:59.548651 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:23:59.549822 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:23:59.549822 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:23:59.549822 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 14:23:59.549822 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 14:23:59.549822 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 14:23:59.549822 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 14:23:59.549822 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 14:23:59.549822 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 14:23:59.549822 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 07 14:23:59.549822 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:23:59.549822 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 626/1267] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:23:59 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:23:59.554636 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324de8aa-d360-4834-b3f7-9b0a9a3439aa None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:59.654573 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324de8aa-d360-4834-b3f7-9b0a9a3439aa None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:59.688401 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50970f79-04e0-43ad-a770-b80f61f3d28f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:59.689179 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50970f79-04e0-43ad-a770-b80f61f3d28f None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:59.689469 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50970f79-04e0-43ad-a770-b80f61f3d28f None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:59.689750 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50970f79-04e0-43ad-a770-b80f61f3d28f None admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:59.690597 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50970f79-04e0-43ad-a770-b80f61f3d28f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:23:59.690912 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50970f79-04e0-43ad-a770-b80f61f3d28f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:59.692400 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50970f79-04e0-43ad-a770-b80f61f3d28f None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:59.701965 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:23:59.709927 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 642/1268] 10.222.0.85 () {56 vars in 1143 bytes} [Tue Jun 7 14:23:59 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:23:59.716272 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bb9760-0ed6-4075-bae9-2ebb3ee85b98 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:59.816588 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bb9760-0ed6-4075-bae9-2ebb3ee85b98 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:23:59.849729 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8671af55-8f3b-4730-9832-32cd0b703376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:23:59.850505 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8671af55-8f3b-4730-9832-32cd0b703376 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:23:59.850785 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8671af55-8f3b-4730-9832-32cd0b703376 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:23:59.851059 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8671af55-8f3b-4730-9832-32cd0b703376 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:23:59.862280 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8671af55-8f3b-4730-9832-32cd0b703376 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:23:59.863704 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8671af55-8f3b-4730-9832-32cd0b703376 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:23:59.880609 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-8671af55-8f3b-4730-9832-32cd0b703376 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:23:59.880609 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:23:59.880609 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:23:59.880609 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:23:59.880609 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:23:59.880609 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:23:59.880609 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:23:59.880609 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:23:59.880609 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:23:59.880609 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:23:59.880609 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:23:59.880609 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:23:59.880609 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:23:59.880609 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:23:59.880609 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:23:59.881661 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:23:59.881661 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:23:59.881661 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:23:59.881661 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:23:59.881661 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:23:59.881661 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:23:59.881661 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:23:59.881661 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:23:59.881661 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:23:59.881661 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:23:59.881661 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:23:59.881661 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:23:59.881661 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:23:59.881661 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:23:59.881661 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:23:59.881661 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:23:59.881661 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:23:59.882804 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:23:59.882804 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:23:59.882804 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:23:59.882804 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:23:59.882804 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 627/1269] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:23:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:23:59.887634 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50970f79-04e0-43ad-a770-b80f61f3d28f None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:23:59.986898 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50970f79-04e0-43ad-a770-b80f61f3d28f None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:24:00.020559 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019abad8-d053-43d7-879a-e1d23c60240c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:24:00.023230 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019abad8-d053-43d7-879a-e1d23c60240c None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:00.023230 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019abad8-d053-43d7-879a-e1d23c60240c None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:00.023230 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019abad8-d053-43d7-879a-e1d23c60240c None admin] PATH_INFO: `/v3/domains` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:00.023230 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019abad8-d053-43d7-879a-e1d23c60240c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:24:00.023230 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019abad8-d053-43d7-879a-e1d23c60240c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:24:00.024130 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019abad8-d053-43d7-879a-e1d23c60240c None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:24:00.035800 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 643/1270] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jun 7 14:23:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:24:00.041727 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8671af55-8f3b-4730-9832-32cd0b703376 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:24:00.074380 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:24:00.140937 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8671af55-8f3b-4730-9832-32cd0b703376 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:24:00.175189 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004b21df-33fb-481f-be04-bb38b94d0491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:24:00.175985 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004b21df-33fb-481f-be04-bb38b94d0491 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:00.176267 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004b21df-33fb-481f-be04-bb38b94d0491 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:00.176575 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004b21df-33fb-481f-be04-bb38b94d0491 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:00.183276 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004b21df-33fb-481f-be04-bb38b94d0491 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:24:00.184760 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004b21df-33fb-481f-be04-bb38b94d0491 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:24:00.199866 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-004b21df-33fb-481f-be04-bb38b94d0491 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:24:00.199866 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:24:00.199866 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:24:00.199866 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:24:00.199866 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:24:00.199866 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:24:00.199866 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:24:00.199866 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:24:00.199866 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:24:00.199866 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:24:00.199866 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:24:00.199866 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:24:00.199866 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 14:24:00.199866 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 14:24:00.199866 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 14:24:00.201038 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 14:24:00.201038 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:24:00.201038 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:24:00.201038 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:24:00.201038 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:24:00.201038 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:24:00.201038 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:24:00.201038 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:24:00.201038 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:24:00.201038 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:24:00.201038 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:24:00.201038 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:24:00.201038 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:24:00.201038 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:24:00.201038 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:24:00.201038 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:24:00.201038 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:24:00.202115 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 14:24:00.202115 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 14:24:00.202115 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 14:24:00.202115 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 14:24:00.202115 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 14:24:00.202115 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 14:24:00.202115 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:24:00.202115 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:24:00.202115 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 628/1271] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jun 7 14:24:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:24:00.206939 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019abad8-d053-43d7-879a-e1d23c60240c None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:24:00.266731 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:24:00.308207 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019abad8-d053-43d7-879a-e1d23c60240c None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:24:00.342826 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f84d369-f08e-4669-8dad-a88736832e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:24:00.343424 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f84d369-f08e-4669-8dad-a88736832e46 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:00.343713 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f84d369-f08e-4669-8dad-a88736832e46 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:00.343996 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f84d369-f08e-4669-8dad-a88736832e46 None admin] PATH_INFO: `/v3/projects` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:00.344818 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f84d369-f08e-4669-8dad-a88736832e46 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:24:00.345134 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f84d369-f08e-4669-8dad-a88736832e46 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:24:00.346635 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f84d369-f08e-4669-8dad-a88736832e46 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:24:00.358379 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 644/1272] 10.222.0.85 () {56 vars in 1148 bytes} [Tue Jun 7 14:24:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:24:00.364489 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004b21df-33fb-481f-be04-bb38b94d0491 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:24:00.462348 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:24:00.471174 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004b21df-33fb-481f-be04-bb38b94d0491 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:24:00.504831 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27ec752-ca8f-4286-b419-01e031b1514d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:24:00.505730 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27ec752-ca8f-4286-b419-01e031b1514d None admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:00.506050 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27ec752-ca8f-4286-b419-01e031b1514d None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:00.506330 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27ec752-ca8f-4286-b419-01e031b1514d None admin] PATH_INFO: `/v3/projects/724785eb74e44daf8736fd01fc0d1839/users/6330294b1c8e4dcd85aad189f007b0aa/roles/9baff2b18f2649f7b757ff2e1f63c853` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:00.508297 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27ec752-ca8f-4286-b419-01e031b1514d None admin] RBAC: Authorizing `identity:create_grant(project_id=724785eb74e44daf8736fd01fc0d1839, user_id=6330294b1c8e4dcd85aad189f007b0aa, role_id=9baff2b18f2649f7b757ff2e1f63c853)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:24:00.510370 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27ec752-ca8f-4286-b419-01e031b1514d None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:24:00.748242 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 629/1273] 10.222.0.85 () {58 vars in 1439 bytes} [Tue Jun 7 14:24:00 2022] PUT /identity/v3/projects/724785eb74e44daf8736fd01fc0d1839/users/6330294b1c8e4dcd85aad189f007b0aa/roles/9baff2b18f2649f7b757ff2e1f63c853 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:24:03.050726 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec9ec00-e4c2-4361-bd58-25a27765b301 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:03.051267 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec9ec00-e4c2-4361-bd58-25a27765b301 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:03.051345 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec9ec00-e4c2-4361-bd58-25a27765b301 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:03.052771 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 645/1274] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:24:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:24:03.058144 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f942980b-ce26-43d6-874c-4bf5cd2d1b2a None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:03.058599 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f942980b-ce26-43d6-874c-4bf5cd2d1b2a None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:03.058781 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f942980b-ce26-43d6-874c-4bf5cd2d1b2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:03.075452 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-f942980b-ce26-43d6-874c-4bf5cd2d1b2a None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:24:03.076715 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-f942980b-ce26-43d6-874c-4bf5cd2d1b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:24:03.150550 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 630/1275] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:24:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:24:03.173612 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec9ec00-e4c2-4361-bd58-25a27765b301 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:24:03.275842 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec9ec00-e4c2-4361-bd58-25a27765b301 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:24:03.294408 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:24:03.311168 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73786ff8-3bb8-4fa6-a552-6e4edc04f840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:24:03.312889 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73786ff8-3bb8-4fa6-a552-6e4edc04f840 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:03.312889 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73786ff8-3bb8-4fa6-a552-6e4edc04f840 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:03.312889 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73786ff8-3bb8-4fa6-a552-6e4edc04f840 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:03.323073 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73786ff8-3bb8-4fa6-a552-6e4edc04f840 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 14:24:03.330732 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-73786ff8-3bb8-4fa6-a552-6e4edc04f840 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:24:03.330732 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:24:03.330732 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:24:03.330732 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:24:03.330732 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:24:03.330732 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:24:03.330732 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:24:03.330732 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:24:03.330732 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:24:03.330732 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:24:03.330732 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:24:03.330732 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:24:03.330732 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 14:24:03.330732 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 14:24:03.330732 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 14:24:03.330732 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 14:24:03.334942 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 14:24:03.334942 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 14:24:03.334942 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 14:24:03.334942 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 14:24:03.334942 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:24:03.334942 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:24:03.334942 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:24:03.334942 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:24:03.334942 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:24:03.334942 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:24:03.334942 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:24:03.334942 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:24:03.334942 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:24:03.334942 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:24:03.334942 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:24:03.334942 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:24:03.335981 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:24:03.335981 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:24:03.335981 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:24:03.335981 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:24:03.335981 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 14:24:03.335981 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 14:24:03.335981 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 14:24:03.335981 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 14:24:03.335981 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 14:24:03.335981 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 14:24:03.335981 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 14:24:03.335981 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:24:03.335981 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 646/1276] 10.222.0.85 () {56 vars in 1096 bytes} [Tue Jun 7 14:24:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:24:03.338065 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f942980b-ce26-43d6-874c-4bf5cd2d1b2a None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:24:03.420512 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:24:03.438566 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f942980b-ce26-43d6-874c-4bf5cd2d1b2a None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:24:03.471650 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29326fa6-ee9e-4e30-9a93-bd21cc7704c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:24:03.472314 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29326fa6-ee9e-4e30-9a93-bd21cc7704c6 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:03.472634 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29326fa6-ee9e-4e30-9a93-bd21cc7704c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:03.472915 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29326fa6-ee9e-4e30-9a93-bd21cc7704c6 None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:03.473661 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29326fa6-ee9e-4e30-9a93-bd21cc7704c6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:24:03.473971 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29326fa6-ee9e-4e30-9a93-bd21cc7704c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:24:03.475334 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29326fa6-ee9e-4e30-9a93-bd21cc7704c6 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:24:03.483283 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 631/1277] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jun 7 14:24:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:24:03.489146 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73786ff8-3bb8-4fa6-a552-6e4edc04f840 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:24:03.592611 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73786ff8-3bb8-4fa6-a552-6e4edc04f840 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:24:03.627815 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ee0c49-df4a-4e16-bb84-566c376613e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:24:03.628633 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ee0c49-df4a-4e16-bb84-566c376613e2 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:03.628922 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ee0c49-df4a-4e16-bb84-566c376613e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:03.629201 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ee0c49-df4a-4e16-bb84-566c376613e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:03.641552 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ee0c49-df4a-4e16-bb84-566c376613e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:24:03.643094 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ee0c49-df4a-4e16-bb84-566c376613e2 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:24:03.660841 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-f2ee0c49-df4a-4e16-bb84-566c376613e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:24:03.660841 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:24:03.660841 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:24:03.660841 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:24:03.660841 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:24:03.660841 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:24:03.660841 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:24:03.660841 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:24:03.660841 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:24:03.660841 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:24:03.660841 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:24:03.660841 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:24:03.660841 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:24:03.660841 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:24:03.660841 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:24:03.661754 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:24:03.661754 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:24:03.661754 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:24:03.661754 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:24:03.661754 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:24:03.661754 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:24:03.661754 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:24:03.661754 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:24:03.661754 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:24:03.661754 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:24:03.661754 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:24:03.661754 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:24:03.661754 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:24:03.661754 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:24:03.661754 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:24:03.661754 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:24:03.661754 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:24:03.662791 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:24:03.662791 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:24:03.662791 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:24:03.662791 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:24:03.662791 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 647/1278] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:24:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:24:03.667786 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29326fa6-ee9e-4e30-9a93-bd21cc7704c6 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:24:03.767543 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29326fa6-ee9e-4e30-9a93-bd21cc7704c6 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:24:03.800711 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a34d91-1716-4cad-a5ac-a3364b7bc35e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:24:03.801364 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a34d91-1716-4cad-a5ac-a3364b7bc35e None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:03.801648 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a34d91-1716-4cad-a5ac-a3364b7bc35e None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:03.801925 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a34d91-1716-4cad-a5ac-a3364b7bc35e None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:03.802651 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a34d91-1716-4cad-a5ac-a3364b7bc35e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:24:03.803152 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a34d91-1716-4cad-a5ac-a3364b7bc35e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:24:03.804537 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a34d91-1716-4cad-a5ac-a3364b7bc35e None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:24:03.818703 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 632/1279] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jun 7 14:24:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:24:03.824452 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ee0c49-df4a-4e16-bb84-566c376613e2 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:24:03.855577 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:24:03.922026 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ee0c49-df4a-4e16-bb84-566c376613e2 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:24:03.956154 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a148402-66b4-4e60-a7d7-45b34cd8a3fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:24:03.956985 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a148402-66b4-4e60-a7d7-45b34cd8a3fd None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:03.957277 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a148402-66b4-4e60-a7d7-45b34cd8a3fd None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:03.957595 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a148402-66b4-4e60-a7d7-45b34cd8a3fd None admin] PATH_INFO: `/v3/users/placement` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:03.964962 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a148402-66b4-4e60-a7d7-45b34cd8a3fd None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:24:03.966401 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a148402-66b4-4e60-a7d7-45b34cd8a3fd None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:24:03.982865 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-1a148402-66b4-4e60-a7d7-45b34cd8a3fd None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 07 14:24:03.982865 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:24:03.982865 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:24:03.982865 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:24:03.982865 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:24:03.982865 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:24:03.982865 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:24:03.982865 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:24:03.982865 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:24:03.982865 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:24:03.982865 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:24:03.982865 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:24:03.982865 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 14:24:03.982865 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 14:24:03.982865 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 14:24:03.982865 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 14:24:03.984021 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:24:03.984021 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:24:03.984021 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 14:24:03.984021 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:24:03.984021 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 14:24:03.984021 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 14:24:03.984021 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:24:03.984021 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:24:03.984021 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:24:03.984021 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:24:03.984021 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:24:03.984021 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:24:03.984021 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:24:03.984021 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:24:03.984021 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:24:03.984021 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:24:03.984021 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:24:03.984021 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:24:03.985380 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:24:03.985380 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:24:03.985380 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 14:24:03.985380 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 14:24:03.985380 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 14:24:03.985380 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 14:24:03.985380 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 14:24:03.985380 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 14:24:03.985380 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 07 14:24:03.985380 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:24:03.985380 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 648/1280] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:24:03 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:24:03.989792 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a34d91-1716-4cad-a5ac-a3364b7bc35e None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:24:03.994095 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:24:04.086134 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a34d91-1716-4cad-a5ac-a3364b7bc35e None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:24:04.121270 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b322e634-0d31-4bad-b48d-f43f8b389ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:24:04.121866 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b322e634-0d31-4bad-b48d-f43f8b389ba2 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:04.122153 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b322e634-0d31-4bad-b48d-f43f8b389ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:04.122433 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b322e634-0d31-4bad-b48d-f43f8b389ba2 None admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:04.123555 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b322e634-0d31-4bad-b48d-f43f8b389ba2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:24:04.123863 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b322e634-0d31-4bad-b48d-f43f8b389ba2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:24:04.125337 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b322e634-0d31-4bad-b48d-f43f8b389ba2 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:24:04.142161 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 633/1281] 10.222.0.85 () {56 vars in 1143 bytes} [Tue Jun 7 14:24:03 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:24:04.148211 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a148402-66b4-4e60-a7d7-45b34cd8a3fd None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:24:04.236536 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:24:04.245829 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a148402-66b4-4e60-a7d7-45b34cd8a3fd None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:24:04.280271 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e443ca8-20bd-40b5-86bb-aa24feb8e1df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:24:04.280953 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e443ca8-20bd-40b5-86bb-aa24feb8e1df None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:04.281240 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e443ca8-20bd-40b5-86bb-aa24feb8e1df None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:04.281519 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e443ca8-20bd-40b5-86bb-aa24feb8e1df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:04.349140 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e443ca8-20bd-40b5-86bb-aa24feb8e1df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:24:04.350605 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e443ca8-20bd-40b5-86bb-aa24feb8e1df None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:24:04.440885 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:24:04.452184 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 649/1282] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jun 7 14:24:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 14:24:04.459542 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b322e634-0d31-4bad-b48d-f43f8b389ba2 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:24:04.529878 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:24:04.568558 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b322e634-0d31-4bad-b48d-f43f8b389ba2 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:24:04.603689 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670e8d2d-42ea-4be8-90f3-3eab408b1252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:24:04.604553 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e8d2d-42ea-4be8-90f3-3eab408b1252 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:04.604872 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e8d2d-42ea-4be8-90f3-3eab408b1252 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:04.605153 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e8d2d-42ea-4be8-90f3-3eab408b1252 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:04.615230 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670e8d2d-42ea-4be8-90f3-3eab408b1252 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:24:04.616700 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670e8d2d-42ea-4be8-90f3-3eab408b1252 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:24:04.635171 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-670e8d2d-42ea-4be8-90f3-3eab408b1252 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:24:04.635171 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:24:04.635171 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:24:04.635171 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:24:04.635171 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:24:04.635171 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:24:04.635171 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:24:04.635171 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:24:04.635171 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:24:04.635171 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:24:04.635171 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:24:04.635171 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:24:04.635171 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 14:24:04.635171 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 14:24:04.635171 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 14:24:04.636296 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 14:24:04.636296 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:24:04.636296 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:24:04.636296 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:24:04.636296 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:24:04.636296 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:24:04.636296 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:24:04.636296 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:24:04.636296 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:24:04.636296 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:24:04.636296 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:24:04.636296 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:24:04.636296 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:24:04.636296 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:24:04.636296 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:24:04.636296 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:24:04.636296 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:24:04.637359 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 14:24:04.637359 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 14:24:04.637359 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 14:24:04.637359 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:24:04.640536 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 634/1283] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:24:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:24:04.642090 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e443ca8-20bd-40b5-86bb-aa24feb8e1df None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:24:04.740660 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e443ca8-20bd-40b5-86bb-aa24feb8e1df None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:24:04.776095 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d91795-a8db-472e-9cf5-a1eac54f977b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:24:04.776788 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d91795-a8db-472e-9cf5-a1eac54f977b None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:04.777080 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d91795-a8db-472e-9cf5-a1eac54f977b None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:04.777365 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d91795-a8db-472e-9cf5-a1eac54f977b None admin] PATH_INFO: `/v3/domains` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:04.778097 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d91795-a8db-472e-9cf5-a1eac54f977b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:24:04.778411 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d91795-a8db-472e-9cf5-a1eac54f977b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:24:04.779777 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d91795-a8db-472e-9cf5-a1eac54f977b None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:24:04.790964 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 650/1284] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jun 7 14:24:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:24:04.796937 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670e8d2d-42ea-4be8-90f3-3eab408b1252 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:24:04.898843 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670e8d2d-42ea-4be8-90f3-3eab408b1252 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:24:04.912789 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:24:04.932642 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3ce43a-82ca-44a3-8b9f-31b12d64515b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:24:04.933431 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3ce43a-82ca-44a3-8b9f-31b12d64515b None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:04.933718 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3ce43a-82ca-44a3-8b9f-31b12d64515b None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:04.933996 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3ce43a-82ca-44a3-8b9f-31b12d64515b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:04.940925 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3ce43a-82ca-44a3-8b9f-31b12d64515b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:24:04.942554 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3ce43a-82ca-44a3-8b9f-31b12d64515b None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:24:04.958138 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-9c3ce43a-82ca-44a3-8b9f-31b12d64515b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:24:04.958138 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:24:04.958138 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:24:04.958138 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:24:04.958138 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:24:04.958138 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:24:04.958138 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:24:04.958138 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:24:04.958138 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:24:04.958138 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:24:04.958138 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:24:04.958138 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:24:04.958138 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 14:24:04.958138 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 14:24:04.958138 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 14:24:04.959063 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 14:24:04.959063 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:24:04.959063 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:24:04.959063 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:24:04.959063 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:24:04.959063 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:24:04.959063 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:24:04.959063 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:24:04.959063 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 07 14:24:04.959063 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:24:04.959063 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:24:04.959063 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:24:04.959063 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:24:04.959063 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:24:04.959063 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:24:04.959063 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:24:04.959063 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:24:04.959999 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 14:24:04.959999 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 14:24:04.959999 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 14:24:04.959999 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 14:24:04.959999 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 14:24:04.959999 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 14:24:04.959999 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 14:24:04.959999 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 07 14:24:04.960972 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 635/1285] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jun 7 14:24:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:24:04.965555 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d91795-a8db-472e-9cf5-a1eac54f977b None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:24:05.016771 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:24:05.064306 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d91795-a8db-472e-9cf5-a1eac54f977b None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:24:05.096724 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228983e4-216a-41d0-b977-a5e8ff5d242f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:24:05.097383 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228983e4-216a-41d0-b977-a5e8ff5d242f None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:05.097675 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228983e4-216a-41d0-b977-a5e8ff5d242f None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:05.097948 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228983e4-216a-41d0-b977-a5e8ff5d242f None admin] PATH_INFO: `/v3/projects` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:05.098734 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228983e4-216a-41d0-b977-a5e8ff5d242f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:24:05.099053 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228983e4-216a-41d0-b977-a5e8ff5d242f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:24:05.100493 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228983e4-216a-41d0-b977-a5e8ff5d242f None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:24:05.116756 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 651/1286] 10.222.0.85 () {56 vars in 1148 bytes} [Tue Jun 7 14:24:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:24:05.118942 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3ce43a-82ca-44a3-8b9f-31b12d64515b None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:24:05.219240 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3ce43a-82ca-44a3-8b9f-31b12d64515b None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:24:05.253070 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abac4406-e0e7-41eb-95dd-fa873d8713e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:24:05.253708 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abac4406-e0e7-41eb-95dd-fa873d8713e0 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:05.253989 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abac4406-e0e7-41eb-95dd-fa873d8713e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:05.254263 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abac4406-e0e7-41eb-95dd-fa873d8713e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:05.255145 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abac4406-e0e7-41eb-95dd-fa873d8713e0 None admin] RBAC: Adding query filter params (role.id=9baff2b18f2649f7b757ff2e1f63c853, scope.project.id=724785eb74e44daf8736fd01fc0d1839, user.id=6330294b1c8e4dcd85aad189f007b0aa) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:24:05.255451 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abac4406-e0e7-41eb-95dd-fa873d8713e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:24:05.256922 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abac4406-e0e7-41eb-95dd-fa873d8713e0 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:24:05.264421 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 636/1287] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jun 7 14:24:05 2022] GET /identity/v3/role_assignments?user.id=6330294b1c8e4dcd85aad189f007b0aa&scope.project.id=724785eb74e44daf8736fd01fc0d1839&role.id=9baff2b18f2649f7b757ff2e1f63c853 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:24:07.665070 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6093736-5f91-4c66-96e0-e6ca0947ff69 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:07.665644 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6093736-5f91-4c66-96e0-e6ca0947ff69 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:07.665644 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6093736-5f91-4c66-96e0-e6ca0947ff69 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:07.667693 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 652/1288] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:24:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 14:24:07.675643 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69e0ba7-b026-40a4-97fc-9446457c23cf None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:07.676085 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69e0ba7-b026-40a4-97fc-9446457c23cf None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:07.676210 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69e0ba7-b026-40a4-97fc-9446457c23cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:07.693598 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-b69e0ba7-b026-40a4-97fc-9446457c23cf None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:24:07.694778 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-b69e0ba7-b026-40a4-97fc-9446457c23cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:24:07.724593 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:24:07.761170 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 637/1289] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:24:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:24:07.785326 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6093736-5f91-4c66-96e0-e6ca0947ff69 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:24:07.889409 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6093736-5f91-4c66-96e0-e6ca0947ff69 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:24:07.900254 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:24:07.924036 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23213ad-89c4-4c59-b8f4-a3bec0e42b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:24:07.924820 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23213ad-89c4-4c59-b8f4-a3bec0e42b4b None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:07.925102 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23213ad-89c4-4c59-b8f4-a3bec0e42b4b None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:07.925377 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23213ad-89c4-4c59-b8f4-a3bec0e42b4b None admin] PATH_INFO: `/v3/services/placement` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:07.926024 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23213ad-89c4-4c59-b8f4-a3bec0e42b4b None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:24:07.927389 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23213ad-89c4-4c59-b8f4-a3bec0e42b4b None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:24:07.940399 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-d23213ad-89c4-4c59-b8f4-a3bec0e42b4b None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 07 14:24:07.940399 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:24:07.940399 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 14:24:07.940399 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 14:24:07.940399 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 14:24:07.940399 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 14:24:07.940399 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 14:24:07.940399 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:24:07.940399 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 07 14:24:07.940399 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:24:07.940399 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 14:24:07.940399 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:24:07.940399 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:24:07.940399 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:24:07.940399 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:24:07.940399 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:24:07.940399 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:24:07.941560 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:24:07.941560 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:24:07.941560 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:24:07.941560 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:24:07.941560 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:24:07.941560 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:24:07.941560 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 14:24:07.941560 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 14:24:07.941560 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 14:24:07.941560 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 14:24:07.941560 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:24:07.941560 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:24:07.941560 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:24:07.941560 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:24:07.941560 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:24:07.941560 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:24:07.942463 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:24:07.942463 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:24:07.942463 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:24:07.942463 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:24:07.942463 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:24:07.942463 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:24:07.942463 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:24:07.942463 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:24:07.942463 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:24:07.942463 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:24:07.942463 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 14:24:07.942463 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:24:07.942463 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 07 14:24:07.942463 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:24:07.942463 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 653/1290] 10.222.0.85 () {56 vars in 1117 bytes} [Tue Jun 7 14:24:07 2022] GET /identity/v3/services/placement => generated 90 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:24:07.948753 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69e0ba7-b026-40a4-97fc-9446457c23cf None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:24:08.065616 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69e0ba7-b026-40a4-97fc-9446457c23cf None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:24:08.102972 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a2e695-0a9b-481c-bc02-90780d74e756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:24:08.103638 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a2e695-0a9b-481c-bc02-90780d74e756 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:08.103923 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a2e695-0a9b-481c-bc02-90780d74e756 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:08.104203 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a2e695-0a9b-481c-bc02-90780d74e756 None admin] PATH_INFO: `/v3/services` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:08.104954 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a2e695-0a9b-481c-bc02-90780d74e756 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:24:08.105271 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a2e695-0a9b-481c-bc02-90780d74e756 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:24:08.106690 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a2e695-0a9b-481c-bc02-90780d74e756 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:24:08.112039 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 638/1291] 10.222.0.85 () {56 vars in 1116 bytes} [Tue Jun 7 14:24:07 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:24:08.118199 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23213ad-89c4-4c59-b8f4-a3bec0e42b4b None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:24:08.223196 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23213ad-89c4-4c59-b8f4-a3bec0e42b4b None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:24:08.257571 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79f871c-2ef2-44fe-8d44-c45a8c53746f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:24:08.258234 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79f871c-2ef2-44fe-8d44-c45a8c53746f None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:08.258516 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79f871c-2ef2-44fe-8d44-c45a8c53746f None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:08.258796 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79f871c-2ef2-44fe-8d44-c45a8c53746f None admin] PATH_INFO: `/v3/services` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:08.259610 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79f871c-2ef2-44fe-8d44-c45a8c53746f None admin] RBAC: Adding query filter params (type=placement) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:24:08.260289 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79f871c-2ef2-44fe-8d44-c45a8c53746f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:24:08.261401 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79f871c-2ef2-44fe-8d44-c45a8c53746f None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:24:08.265895 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 654/1292] 10.222.0.85 () {56 vars in 1116 bytes} [Tue Jun 7 14:24:08 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:24:10.507955 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45d54bd-4fbb-42b7-a503-ef0a88c94977 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:10.508519 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45d54bd-4fbb-42b7-a503-ef0a88c94977 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:10.508617 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45d54bd-4fbb-42b7-a503-ef0a88c94977 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:10.510463 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 639/1293] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:24:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 14:24:10.520654 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146aaf9-271d-4ac8-8de4-1f17ba96b922 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:10.521118 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146aaf9-271d-4ac8-8de4-1f17ba96b922 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:10.521221 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146aaf9-271d-4ac8-8de4-1f17ba96b922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:10.539471 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-4146aaf9-271d-4ac8-8de4-1f17ba96b922 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:24:10.543118 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-4146aaf9-271d-4ac8-8de4-1f17ba96b922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:24:10.590114 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:24:10.627106 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 655/1294] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:24:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:24:10.650287 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45d54bd-4fbb-42b7-a503-ef0a88c94977 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:24:10.683338 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:24:10.759242 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45d54bd-4fbb-42b7-a503-ef0a88c94977 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:24:10.794658 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acf50fe-e62b-49c1-a171-59ae85a2c2de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:24:10.795320 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acf50fe-e62b-49c1-a171-59ae85a2c2de None admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:10.795607 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acf50fe-e62b-49c1-a171-59ae85a2c2de None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:10.795884 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acf50fe-e62b-49c1-a171-59ae85a2c2de None admin] PATH_INFO: `/v3/services` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:10.796685 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acf50fe-e62b-49c1-a171-59ae85a2c2de None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:24:10.798040 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acf50fe-e62b-49c1-a171-59ae85a2c2de None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:24:10.885270 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 640/1295] 10.222.0.85 () {60 vars in 1141 bytes} [Tue Jun 7 14:24:10 2022] POST /identity/v3/services => generated 246 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:24:13.171462 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875439c2-f812-4ac0-ba17-5815f9c08955 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:13.172017 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875439c2-f812-4ac0-ba17-5815f9c08955 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:13.172017 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875439c2-f812-4ac0-ba17-5815f9c08955 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:13.173455 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 656/1296] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:24:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:24:13.178631 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57218060-62f8-4eea-a1cc-7da54a674ac6 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:13.179022 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57218060-62f8-4eea-a1cc-7da54a674ac6 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:13.179190 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57218060-62f8-4eea-a1cc-7da54a674ac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:13.197125 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-57218060-62f8-4eea-a1cc-7da54a674ac6 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:24:13.198318 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-57218060-62f8-4eea-a1cc-7da54a674ac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:24:13.273083 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 641/1297] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:24:13 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 14:24:13.296441 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875439c2-f812-4ac0-ba17-5815f9c08955 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:24:13.404998 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875439c2-f812-4ac0-ba17-5815f9c08955 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:24:13.440206 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7a706e-667d-4101-98ce-c837537ce0a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:24:13.441032 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7a706e-667d-4101-98ce-c837537ce0a1 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:13.441312 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7a706e-667d-4101-98ce-c837537ce0a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:13.441580 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7a706e-667d-4101-98ce-c837537ce0a1 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:13.442231 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7a706e-667d-4101-98ce-c837537ce0a1 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:24:13.443699 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7a706e-667d-4101-98ce-c837537ce0a1 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:24:13.456272 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-6f7a706e-667d-4101-98ce-c837537ce0a1 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 07 14:24:13.456272 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:24:13.456272 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 14:24:13.456272 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 14:24:13.456272 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 14:24:13.456272 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 14:24:13.456272 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 14:24:13.456272 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:24:13.456272 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 07 14:24:13.456272 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:24:13.456272 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 14:24:13.456272 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:24:13.456272 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:24:13.456272 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:24:13.456272 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:24:13.456272 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:24:13.456272 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:24:13.461447 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:24:13.461447 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:24:13.461447 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:24:13.461447 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:24:13.461447 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:24:13.461447 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:24:13.461447 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 14:24:13.461447 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 14:24:13.461447 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 14:24:13.461447 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 14:24:13.461447 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:24:13.461447 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:24:13.461447 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:24:13.461447 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:24:13.461447 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:24:13.461447 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:24:13.463047 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:24:13.463047 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:24:13.463047 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:24:13.463047 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:24:13.463047 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:24:13.463047 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:24:13.463047 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:24:13.463047 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:24:13.463047 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:24:13.463047 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:24:13.463047 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 14:24:13.463047 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:24:13.463047 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 07 14:24:13.463047 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:24:13.463047 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 657/1298] 10.222.0.85 () {56 vars in 1117 bytes} [Tue Jun 7 14:24:13 2022] GET /identity/v3/services/placement => generated 90 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:24:13.464822 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57218060-62f8-4eea-a1cc-7da54a674ac6 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:24:13.485650 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:24:13.570741 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57218060-62f8-4eea-a1cc-7da54a674ac6 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:24:13.605773 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b041441a-f8b1-48cf-a130-6e60338f3d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:24:13.606435 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b041441a-f8b1-48cf-a130-6e60338f3d97 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:13.606720 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b041441a-f8b1-48cf-a130-6e60338f3d97 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:13.606999 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b041441a-f8b1-48cf-a130-6e60338f3d97 None admin] PATH_INFO: `/v3/services` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:13.607712 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b041441a-f8b1-48cf-a130-6e60338f3d97 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:24:13.608023 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b041441a-f8b1-48cf-a130-6e60338f3d97 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:24:13.609473 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b041441a-f8b1-48cf-a130-6e60338f3d97 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:24:13.614967 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 642/1299] 10.222.0.85 () {56 vars in 1116 bytes} [Tue Jun 7 14:24:13 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:24:13.621225 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7a706e-667d-4101-98ce-c837537ce0a1 None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:24:13.654112 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:24:13.721474 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7a706e-667d-4101-98ce-c837537ce0a1 None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:24:13.756061 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd2ce24-02e0-4d18-961b-fcabbefeab72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:24:13.756798 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd2ce24-02e0-4d18-961b-fcabbefeab72 None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:13.757165 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd2ce24-02e0-4d18-961b-fcabbefeab72 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:13.757394 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd2ce24-02e0-4d18-961b-fcabbefeab72 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:13.758126 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd2ce24-02e0-4d18-961b-fcabbefeab72 None admin] RBAC: Adding query filter params (interface=public, service_id=2278be75843b4b48a53086c1f5f40b5b, region_id=RegionOne) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:24:13.758454 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd2ce24-02e0-4d18-961b-fcabbefeab72 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:24:13.760317 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd2ce24-02e0-4d18-961b-fcabbefeab72 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:24:13.766190 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 658/1300] 10.222.0.85 () {56 vars in 1251 bytes} [Tue Jun 7 14:24:13 2022] GET /identity/v3/endpoints?service_id=2278be75843b4b48a53086c1f5f40b5b&interface=public®ion_id=RegionOne => generated 194 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:24:13.771913 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b041441a-f8b1-48cf-a130-6e60338f3d97 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:24:13.853676 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:24:13.872258 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b041441a-f8b1-48cf-a130-6e60338f3d97 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:24:13.907211 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311fae46-bc5c-4d6f-9ee4-c230cc32eff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:24:13.907867 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311fae46-bc5c-4d6f-9ee4-c230cc32eff3 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:13.908149 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311fae46-bc5c-4d6f-9ee4-c230cc32eff3 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:13.908448 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311fae46-bc5c-4d6f-9ee4-c230cc32eff3 None admin] PATH_INFO: `/v3/services` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:13.909127 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311fae46-bc5c-4d6f-9ee4-c230cc32eff3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:24:13.910482 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311fae46-bc5c-4d6f-9ee4-c230cc32eff3 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:24:13.917213 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 643/1301] 10.222.0.85 () {56 vars in 1087 bytes} [Tue Jun 7 14:24:13 2022] GET /identity/v3/services => generated 2191 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:24:16.271163 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f3ceee-1737-4da6-be5f-219e2d42491b None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:16.271689 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f3ceee-1737-4da6-be5f-219e2d42491b None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:16.271689 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f3ceee-1737-4da6-be5f-219e2d42491b None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:16.273105 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 659/1302] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:24:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:24:16.278799 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1956ca-bcab-44ab-a0a7-7fe06014f1c8 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:16.279238 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1956ca-bcab-44ab-a0a7-7fe06014f1c8 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:16.279392 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1956ca-bcab-44ab-a0a7-7fe06014f1c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:16.295925 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-dd1956ca-bcab-44ab-a0a7-7fe06014f1c8 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:24:16.297148 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-dd1956ca-bcab-44ab-a0a7-7fe06014f1c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:24:16.365209 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 644/1303] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:24:16 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 14:24:16.388485 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f3ceee-1737-4da6-be5f-219e2d42491b None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:24:16.484418 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:24:16.490898 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f3ceee-1737-4da6-be5f-219e2d42491b None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:24:16.523884 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6c90ac-87bf-444b-80e8-e8d83670643c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:24:16.525241 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6c90ac-87bf-444b-80e8-e8d83670643c None admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:16.525561 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6c90ac-87bf-444b-80e8-e8d83670643c None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:16.525845 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6c90ac-87bf-444b-80e8-e8d83670643c None admin] PATH_INFO: `/v3/services/placement` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:16.526495 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6c90ac-87bf-444b-80e8-e8d83670643c None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:24:16.527860 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6c90ac-87bf-444b-80e8-e8d83670643c None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:24:16.539949 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application [None req-ce6c90ac-87bf-444b-80e8-e8d83670643c None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 07 14:24:16.539949 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:24:16.539949 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 14:24:16.539949 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 14:24:16.539949 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 14:24:16.539949 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 14:24:16.539949 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 14:24:16.539949 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:24:16.539949 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 07 14:24:16.539949 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:24:16.539949 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 14:24:16.539949 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:24:16.539949 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 14:24:16.539949 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 14:24:16.539949 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 14:24:16.539949 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 14:24:16.539949 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 14:24:16.543528 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 14:24:16.543528 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 14:24:16.543528 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 14:24:16.543528 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 14:24:16.543528 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 14:24:16.543528 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 14:24:16.543528 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 14:24:16.543528 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 14:24:16.543528 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 14:24:16.543528 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 14:24:16.543528 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 14:24:16.543528 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 14:24:16.543528 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 14:24:16.543528 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 14:24:16.543528 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 14:24:16.543528 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 14:24:16.544573 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 14:24:16.544573 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application with Lock( Jun 07 14:24:16.544573 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 14:24:16.544573 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self._enter() Jun 07 14:24:16.544573 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 14:24:16.544573 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 14:24:16.544573 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 14:24:16.544573 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application return self.creator() Jun 07 14:24:16.544573 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 14:24:16.544573 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application created_value = creator( Jun 07 14:24:16.544573 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 14:24:16.544573 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 14:24:16.544573 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 07 14:24:16.544573 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: ERROR keystone.server.flask.application Jun 07 14:24:16.544573 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 660/1304] 10.222.0.85 () {56 vars in 1117 bytes} [Tue Jun 7 14:24:16 2022] GET /identity/v3/services/placement => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 14:24:16.547394 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1956ca-bcab-44ab-a0a7-7fe06014f1c8 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:24:16.625176 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:24:16.651341 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1956ca-bcab-44ab-a0a7-7fe06014f1c8 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:24:16.684720 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2aac38-8f24-4c47-a22d-b6f56d30dd53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:24:16.685400 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2aac38-8f24-4c47-a22d-b6f56d30dd53 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:16.685844 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2aac38-8f24-4c47-a22d-b6f56d30dd53 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:16.686171 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2aac38-8f24-4c47-a22d-b6f56d30dd53 None admin] PATH_INFO: `/v3/services` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:16.687268 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2aac38-8f24-4c47-a22d-b6f56d30dd53 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:24:16.687580 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2aac38-8f24-4c47-a22d-b6f56d30dd53 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:24:16.688996 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2aac38-8f24-4c47-a22d-b6f56d30dd53 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:24:16.694166 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 645/1305] 10.222.0.85 () {56 vars in 1116 bytes} [Tue Jun 7 14:24:16 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:24:16.700420 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6c90ac-87bf-444b-80e8-e8d83670643c None admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:24:16.799676 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6c90ac-87bf-444b-80e8-e8d83670643c None admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:24:16.831983 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16018ba-2d4c-4636-af05-992d311cff47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:24:16.832690 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16018ba-2d4c-4636-af05-992d311cff47 None admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:24:16.832971 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16018ba-2d4c-4636-af05-992d311cff47 None admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:24:16.833245 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16018ba-2d4c-4636-af05-992d311cff47 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:24:16.833968 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16018ba-2d4c-4636-af05-992d311cff47 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:24:16.835347 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16018ba-2d4c-4636-af05-992d311cff47 None admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:24:16.908128 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 661/1306] 10.222.0.85 () {60 vars in 1144 bytes} [Tue Jun 7 14:24:16 2022] POST /identity/v3/endpoints => generated 331 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:24:26.691085 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=30 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:24:26.903557 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:30:37.077181 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797dd465-288b-4506-83f4-f84102fdb944 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:30:37.077730 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797dd465-288b-4506-83f4-f84102fdb944 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:30:37.077730 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797dd465-288b-4506-83f4-f84102fdb944 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:30:37.084485 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e4e93c-ee0d-4ba6-854b-380a7cb64bc7 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:30:37.085081 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e4e93c-ee0d-4ba6-854b-380a7cb64bc7 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:30:37.085081 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e4e93c-ee0d-4ba6-854b-380a7cb64bc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:30:37.088544 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 646/1307] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:30:37 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:30:37.170503 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-11e4e93c-ee0d-4ba6-854b-380a7cb64bc7 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:30:37.171643 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-11e4e93c-ee0d-4ba6-854b-380a7cb64bc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:30:37.328848 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 662/1308] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:30:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:30:37.378037 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23955e3-7948-4166-9bc9-b77a00013aa9 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:30:37.378256 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23955e3-7948-4166-9bc9-b77a00013aa9 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:30:37.378537 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23955e3-7948-4166-9bc9-b77a00013aa9 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:30:37.380252 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 647/1309] 10.222.0.85 () {54 vars in 964 bytes} [Tue Jun 7 14:30:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:30:37.387023 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a072aa9c-50b9-4fe5-a8c7-f4a8f7c5be91 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:30:37.387331 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a072aa9c-50b9-4fe5-a8c7-f4a8f7c5be91 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:30:37.387619 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a072aa9c-50b9-4fe5-a8c7-f4a8f7c5be91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:30:37.550642 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-a072aa9c-50b9-4fe5-a8c7-f4a8f7c5be91 None None] MFA Rules not processed for user `7ac4c85918c045b793a1a53300c989a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:30:37.551929 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-a072aa9c-50b9-4fe5-a8c7-f4a8f7c5be91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:30:37.617033 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:30:37.687524 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 663/1310] 10.222.0.85 () {58 vars in 1061 bytes} [Tue Jun 7 14:30:37 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:30:37.696533 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23955e3-7948-4166-9bc9-b77a00013aa9 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:30:37.699292 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-f23955e3-7948-4166-9bc9-b77a00013aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:30:37.833074 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23955e3-7948-4166-9bc9-b77a00013aa9 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:30:37.917099 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c61b783-16e1-4aeb-9431-153b288288c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:30:37.917862 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c61b783-16e1-4aeb-9431-153b288288c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:30:37.918257 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c61b783-16e1-4aeb-9431-153b288288c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:30:37.918641 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c61b783-16e1-4aeb-9431-153b288288c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:30:37.920556 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-2c61b783-16e1-4aeb-9431-153b288288c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:30:37.982743 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:30:38.044172 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c61b783-16e1-4aeb-9431-153b288288c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:30:38.045871 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c61b783-16e1-4aeb-9431-153b288288c1 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:30:38.066683 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 648/1311] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:30:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:30:40.793559 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4384c20f-72b7-43ea-bdc9-e1fe79a35627 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:30:40.794297 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4384c20f-72b7-43ea-bdc9-e1fe79a35627 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:30:40.794443 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4384c20f-72b7-43ea-bdc9-e1fe79a35627 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:30:40.796549 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 664/1312] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:30:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 14:30:40.802049 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d79afab-e4c8-477f-84d2-fa2957a2e30b None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:30:40.802585 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d79afab-e4c8-477f-84d2-fa2957a2e30b None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:30:40.802585 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d79afab-e4c8-477f-84d2-fa2957a2e30b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:30:40.901681 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-8d79afab-e4c8-477f-84d2-fa2957a2e30b None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:30:40.902919 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-8d79afab-e4c8-477f-84d2-fa2957a2e30b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:30:41.012238 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 649/1313] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:30:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:30:41.050968 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4384c20f-72b7-43ea-bdc9-e1fe79a35627 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:30:41.052650 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-4384c20f-72b7-43ea-bdc9-e1fe79a35627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:30:41.163190 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4384c20f-72b7-43ea-bdc9-e1fe79a35627 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:30:41.238806 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:30:41.241356 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c4be5e-a308-4629-8af5-41e055a3308d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:30:41.242210 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c4be5e-a308-4629-8af5-41e055a3308d service neutron] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:30:41.242492 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c4be5e-a308-4629-8af5-41e055a3308d service neutron] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:30:41.242766 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c4be5e-a308-4629-8af5-41e055a3308d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:30:41.244487 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-d2c4be5e-a308-4629-8af5-41e055a3308d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:30:41.375660 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c4be5e-a308-4629-8af5-41e055a3308d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:30:41.377143 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c4be5e-a308-4629-8af5-41e055a3308d service neutron] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:30:41.411013 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 665/1314] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:30:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:30:44.163662 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc8d6c7-82bb-4af4-9cce-1fa566ee84e2 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:30:44.164307 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc8d6c7-82bb-4af4-9cce-1fa566ee84e2 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:30:44.164307 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc8d6c7-82bb-4af4-9cce-1fa566ee84e2 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:30:44.165630 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 650/1315] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:30:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:30:44.171673 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1da41b4-d811-449e-8640-19bb7aaaca3a None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:30:44.172097 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1da41b4-d811-449e-8640-19bb7aaaca3a None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:30:44.172230 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1da41b4-d811-449e-8640-19bb7aaaca3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:30:44.235382 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-c1da41b4-d811-449e-8640-19bb7aaaca3a None None] MFA Rules not processed for user `37e5547faf8a4aebb8725f07a28cd760`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:30:44.236538 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-c1da41b4-d811-449e-8640-19bb7aaaca3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:30:44.287070 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:30:44.330746 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 666/1316] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:30:44 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:30:44.370797 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc8d6c7-82bb-4af4-9cce-1fa566ee84e2 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:30:44.372341 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-efc8d6c7-82bb-4af4-9cce-1fa566ee84e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:30:44.382508 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:30:44.481355 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc8d6c7-82bb-4af4-9cce-1fa566ee84e2 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:30:44.556720 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cf49f6-5119-47a9-86db-3839bafb92d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:30:44.557302 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf49f6-5119-47a9-86db-3839bafb92d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:30:44.557583 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf49f6-5119-47a9-86db-3839bafb92d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:30:44.557857 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf49f6-5119-47a9-86db-3839bafb92d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:30:44.559495 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-94cf49f6-5119-47a9-86db-3839bafb92d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:30:44.619830 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:30:44.666464 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cf49f6-5119-47a9-86db-3839bafb92d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:30:44.667896 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cf49f6-5119-47a9-86db-3839bafb92d9 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:30:44.686349 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 651/1317] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:30:44 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:30:47.724444 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffde59c-8dba-4704-8408-f764416dc9dd None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:30:47.725037 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffde59c-8dba-4704-8408-f764416dc9dd None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:30:47.725037 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffde59c-8dba-4704-8408-f764416dc9dd None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:30:47.726393 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 667/1318] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:30:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:30:47.731575 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf4ac54-eeca-47a5-836b-0ccb08cc9ea5 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:30:47.731970 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf4ac54-eeca-47a5-836b-0ccb08cc9ea5 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:30:47.732192 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf4ac54-eeca-47a5-836b-0ccb08cc9ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:30:47.796831 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-8bf4ac54-eeca-47a5-836b-0ccb08cc9ea5 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:30:47.798042 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-8bf4ac54-eeca-47a5-836b-0ccb08cc9ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:30:47.898475 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 652/1319] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:30:47 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:30:47.938181 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffde59c-8dba-4704-8408-f764416dc9dd None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:30:47.939713 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-2ffde59c-8dba-4704-8408-f764416dc9dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:30:48.041785 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffde59c-8dba-4704-8408-f764416dc9dd None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:30:48.111821 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d707c262-5651-4220-8148-4e6e5f2ff745 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:30:48.112488 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d707c262-5651-4220-8148-4e6e5f2ff745 service neutron] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:30:48.112771 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d707c262-5651-4220-8148-4e6e5f2ff745 service neutron] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:30:48.113046 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d707c262-5651-4220-8148-4e6e5f2ff745 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:30:48.114668 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-d707c262-5651-4220-8148-4e6e5f2ff745 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:30:48.122391 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:30:48.226614 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d707c262-5651-4220-8148-4e6e5f2ff745 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:30:48.228050 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d707c262-5651-4220-8148-4e6e5f2ff745 service neutron] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:30:48.246733 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 668/1320] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:30:47 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:30:51.469269 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcbb0a9-9fa1-45da-9939-13283b51ea6d None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:30:51.469773 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcbb0a9-9fa1-45da-9939-13283b51ea6d None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:30:51.469865 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcbb0a9-9fa1-45da-9939-13283b51ea6d None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:30:51.471372 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 653/1321] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:30:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:30:51.476583 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c045d4c6-cabb-44a0-90f8-fe19aa158543 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:30:51.477012 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c045d4c6-cabb-44a0-90f8-fe19aa158543 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:30:51.477158 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c045d4c6-cabb-44a0-90f8-fe19aa158543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:30:51.543377 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-c045d4c6-cabb-44a0-90f8-fe19aa158543 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:30:51.544637 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-c045d4c6-cabb-44a0-90f8-fe19aa158543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:30:51.602068 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:30:51.640862 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 669/1322] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:30:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:30:51.677326 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcbb0a9-9fa1-45da-9939-13283b51ea6d None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:30:51.678876 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-5fcbb0a9-9fa1-45da-9939-13283b51ea6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:30:51.697971 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:30:51.782992 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcbb0a9-9fa1-45da-9939-13283b51ea6d None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:30:51.856307 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284e59e6-c1ff-4ca8-af4d-b7ff5db851e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:30:51.856992 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284e59e6-c1ff-4ca8-af4d-b7ff5db851e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:30:51.857280 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284e59e6-c1ff-4ca8-af4d-b7ff5db851e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:30:51.857579 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284e59e6-c1ff-4ca8-af4d-b7ff5db851e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:30:51.859217 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-284e59e6-c1ff-4ca8-af4d-b7ff5db851e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:30:51.931395 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:30:51.976196 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284e59e6-c1ff-4ca8-af4d-b7ff5db851e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:30:51.976196 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284e59e6-c1ff-4ca8-af4d-b7ff5db851e0 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:30:51.994423 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 654/1323] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:30:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:30:54.336629 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c5afa6-fa3b-4d07-8dc3-372d8659a5ce None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:30:54.337136 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c5afa6-fa3b-4d07-8dc3-372d8659a5ce None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:30:54.337136 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c5afa6-fa3b-4d07-8dc3-372d8659a5ce None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:30:54.338510 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 670/1324] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:30:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:30:54.344511 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01d055b-c60b-4137-bed5-f068acb924b5 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:30:54.344949 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01d055b-c60b-4137-bed5-f068acb924b5 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:30:54.345076 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01d055b-c60b-4137-bed5-f068acb924b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:30:54.409426 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-b01d055b-c60b-4137-bed5-f068acb924b5 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:30:54.410643 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-b01d055b-c60b-4137-bed5-f068acb924b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:30:54.511741 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 655/1325] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:30:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:30:54.548095 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c5afa6-fa3b-4d07-8dc3-372d8659a5ce None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:30:54.549740 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-13c5afa6-fa3b-4d07-8dc3-372d8659a5ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:30:54.662898 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c5afa6-fa3b-4d07-8dc3-372d8659a5ce None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:30:54.739301 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5696d73e-398b-4c66-9d06-8820888d59ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:30:54.739990 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5696d73e-398b-4c66-9d06-8820888d59ae service neutron] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:30:54.740271 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5696d73e-398b-4c66-9d06-8820888d59ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:30:54.740573 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5696d73e-398b-4c66-9d06-8820888d59ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:30:54.742223 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-5696d73e-398b-4c66-9d06-8820888d59ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:30:54.759291 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:30:54.871815 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5696d73e-398b-4c66-9d06-8820888d59ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:30:54.873172 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5696d73e-398b-4c66-9d06-8820888d59ae service neutron] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:30:54.891609 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 671/1326] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:30:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:30:57.204329 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4802680e-ea62-4ec3-8815-8791061c07bd None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:30:57.204861 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4802680e-ea62-4ec3-8815-8791061c07bd None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:30:57.204975 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4802680e-ea62-4ec3-8815-8791061c07bd None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:30:57.206299 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 656/1327] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:30:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:30:57.211521 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeb0b30-171b-4fed-860f-09f295817bca None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:30:57.211945 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeb0b30-171b-4fed-860f-09f295817bca None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:30:57.212040 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeb0b30-171b-4fed-860f-09f295817bca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:30:57.278624 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-eaeb0b30-171b-4fed-860f-09f295817bca None None] MFA Rules not processed for user `37e5547faf8a4aebb8725f07a28cd760`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:30:57.279845 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-eaeb0b30-171b-4fed-860f-09f295817bca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:30:57.361166 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:30:57.392986 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 672/1328] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:30:57 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:30:57.429664 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4802680e-ea62-4ec3-8815-8791061c07bd None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:30:57.431579 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-4802680e-ea62-4ec3-8815-8791061c07bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:30:57.466372 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:30:57.542268 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4802680e-ea62-4ec3-8815-8791061c07bd None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:30:57.617157 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f96604-549e-4acc-bb03-3b56df4421c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:30:57.617808 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f96604-549e-4acc-bb03-3b56df4421c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:30:57.618090 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f96604-549e-4acc-bb03-3b56df4421c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:30:57.618361 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f96604-549e-4acc-bb03-3b56df4421c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:30:57.620008 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-84f96604-549e-4acc-bb03-3b56df4421c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:30:57.698257 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:30:57.729860 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f96604-549e-4acc-bb03-3b56df4421c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:30:57.732589 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f96604-549e-4acc-bb03-3b56df4421c2 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:30:57.751581 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 657/1329] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:30:57 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 14:31:00.549048 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff75e30-1c48-4a4b-b97d-85cc1cce2b52 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:31:00.549539 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff75e30-1c48-4a4b-b97d-85cc1cce2b52 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:31:00.549539 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff75e30-1c48-4a4b-b97d-85cc1cce2b52 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:31:00.550951 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 673/1330] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:31:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:31:00.556511 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa5cc67-7bbe-4d82-bc7c-492cf4c9cb4a None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:31:00.557008 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa5cc67-7bbe-4d82-bc7c-492cf4c9cb4a None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:31:00.557102 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa5cc67-7bbe-4d82-bc7c-492cf4c9cb4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:31:00.625111 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-aaa5cc67-7bbe-4d82-bc7c-492cf4c9cb4a None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:31:00.626435 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-aaa5cc67-7bbe-4d82-bc7c-492cf4c9cb4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:31:00.725308 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 658/1331] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:31:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:31:00.763162 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff75e30-1c48-4a4b-b97d-85cc1cce2b52 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:31:00.764785 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-2ff75e30-1c48-4a4b-b97d-85cc1cce2b52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:31:00.872812 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff75e30-1c48-4a4b-b97d-85cc1cce2b52 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:31:00.954286 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3e0fe3-0b98-4131-bbc6-acd4d8ff7fd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:31:00.954960 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3e0fe3-0b98-4131-bbc6-acd4d8ff7fd1 service neutron] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:31:00.955244 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3e0fe3-0b98-4131-bbc6-acd4d8ff7fd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:31:00.955523 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3e0fe3-0b98-4131-bbc6-acd4d8ff7fd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:31:00.957283 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-ac3e0fe3-0b98-4131-bbc6-acd4d8ff7fd1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:31:00.991249 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:31:01.080679 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3e0fe3-0b98-4131-bbc6-acd4d8ff7fd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:31:01.082123 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3e0fe3-0b98-4131-bbc6-acd4d8ff7fd1 service neutron] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:31:01.105479 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 674/1332] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:31:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:31:03.442020 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385cbbd2-8f6f-4a6a-9e61-0ffa4afe70ae None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:31:03.442629 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385cbbd2-8f6f-4a6a-9e61-0ffa4afe70ae None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:31:03.442629 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385cbbd2-8f6f-4a6a-9e61-0ffa4afe70ae None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:31:03.443957 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 659/1333] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:31:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:31:03.450717 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b843dccd-fdfa-4e04-8857-f00d176ee01c None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:31:03.451207 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b843dccd-fdfa-4e04-8857-f00d176ee01c None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:31:03.451682 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b843dccd-fdfa-4e04-8857-f00d176ee01c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:31:03.536820 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-b843dccd-fdfa-4e04-8857-f00d176ee01c None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:31:03.538157 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-b843dccd-fdfa-4e04-8857-f00d176ee01c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:31:03.628693 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:31:03.660432 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 675/1334] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:31:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:31:03.717470 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607d6ce3-d4bc-4f7b-b459-c1a6d8c3a6af None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:31:03.717768 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607d6ce3-d4bc-4f7b-b459-c1a6d8c3a6af None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:31:03.718057 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607d6ce3-d4bc-4f7b-b459-c1a6d8c3a6af None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:31:03.719421 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 660/1335] 10.222.0.85 () {54 vars in 964 bytes} [Tue Jun 7 14:31:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:31:03.725477 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea6f551-797d-47d4-ad6b-326feb3e0b8a None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:31:03.725776 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea6f551-797d-47d4-ad6b-326feb3e0b8a None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:31:03.726053 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea6f551-797d-47d4-ad6b-326feb3e0b8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:31:03.806276 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-8ea6f551-797d-47d4-ad6b-326feb3e0b8a None None] MFA Rules not processed for user `7ac4c85918c045b793a1a53300c989a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:31:03.807509 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-8ea6f551-797d-47d4-ad6b-326feb3e0b8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:31:03.886873 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 676/1336] 10.222.0.85 () {58 vars in 1061 bytes} [Tue Jun 7 14:31:03 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:31:03.896181 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607d6ce3-d4bc-4f7b-b459-c1a6d8c3a6af None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:31:03.897812 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-607d6ce3-d4bc-4f7b-b459-c1a6d8c3a6af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:31:03.944220 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:31:04.019276 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607d6ce3-d4bc-4f7b-b459-c1a6d8c3a6af None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:31:04.236574 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc934d4f-813b-4cfa-b795-d322e6a9b0a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:31:04.237144 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc934d4f-813b-4cfa-b795-d322e6a9b0a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:31:04.237429 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc934d4f-813b-4cfa-b795-d322e6a9b0a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:31:04.237702 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc934d4f-813b-4cfa-b795-d322e6a9b0a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:31:04.239389 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-bc934d4f-813b-4cfa-b795-d322e6a9b0a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:31:04.321194 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:31:04.347173 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc934d4f-813b-4cfa-b795-d322e6a9b0a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:31:04.348937 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc934d4f-813b-4cfa-b795-d322e6a9b0a1 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:31:04.366672 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 661/1337] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:31:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:31:07.671612 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c66f082-8419-4ed6-a64d-3698fed094da None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:31:07.672178 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c66f082-8419-4ed6-a64d-3698fed094da None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:31:07.672178 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c66f082-8419-4ed6-a64d-3698fed094da None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:31:07.673627 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 677/1338] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:31:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:31:07.678917 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7e8698-fd4a-401b-bce4-82997cff6993 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:31:07.679349 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7e8698-fd4a-401b-bce4-82997cff6993 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:31:07.679462 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7e8698-fd4a-401b-bce4-82997cff6993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:31:07.745906 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-1c7e8698-fd4a-401b-bce4-82997cff6993 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:31:07.747111 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-1c7e8698-fd4a-401b-bce4-82997cff6993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:31:07.854072 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 662/1339] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:31:07 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:31:07.890584 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c66f082-8419-4ed6-a64d-3698fed094da None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:31:07.895754 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-5c66f082-8419-4ed6-a64d-3698fed094da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:31:07.973280 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:31:08.002575 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c66f082-8419-4ed6-a64d-3698fed094da None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:31:08.006156 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539c3727-adaa-4d73-b629-f3aa8fa43882 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:31:08.007071 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539c3727-adaa-4d73-b629-f3aa8fa43882 service neutron] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:31:08.007359 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539c3727-adaa-4d73-b629-f3aa8fa43882 service neutron] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:31:08.007640 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539c3727-adaa-4d73-b629-f3aa8fa43882 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:31:08.037643 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539c3727-adaa-4d73-b629-f3aa8fa43882 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:31:08.039065 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539c3727-adaa-4d73-b629-f3aa8fa43882 service neutron] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:31:08.048603 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 678/1340] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:31:07 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:31:12.746652 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3586a496-c14a-4b8a-8d40-80fa996b5e18 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:31:12.746652 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3586a496-c14a-4b8a-8d40-80fa996b5e18 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:31:12.747292 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3586a496-c14a-4b8a-8d40-80fa996b5e18 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:31:12.748461 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 663/1341] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:31:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:31:12.754388 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c31a7d7-de34-40d2-b73e-a833d77c59e8 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:31:12.754388 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c31a7d7-de34-40d2-b73e-a833d77c59e8 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:31:12.755158 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c31a7d7-de34-40d2-b73e-a833d77c59e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:31:12.827636 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-8c31a7d7-de34-40d2-b73e-a833d77c59e8 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:31:12.828808 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-8c31a7d7-de34-40d2-b73e-a833d77c59e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:31:12.906248 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 679/1342] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:31:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:31:12.945300 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3586a496-c14a-4b8a-8d40-80fa996b5e18 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:31:12.958033 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3586a496-c14a-4b8a-8d40-80fa996b5e18 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:31:12.962156 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b5b88d-590e-4c0a-a885-7f23deed7a1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:31:12.962471 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b5b88d-590e-4c0a-a885-7f23deed7a1f service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:31:12.962756 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b5b88d-590e-4c0a-a885-7f23deed7a1f service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:31:12.963177 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b5b88d-590e-4c0a-a885-7f23deed7a1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:31:12.976479 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b5b88d-590e-4c0a-a885-7f23deed7a1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:31:12.977924 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b5b88d-590e-4c0a-a885-7f23deed7a1f service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:31:12.987883 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 664/1343] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:31:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:31:16.298926 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc27a292-b859-4137-89c5-f2e888f0f187 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:31:16.299484 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc27a292-b859-4137-89c5-f2e888f0f187 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:31:16.299484 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc27a292-b859-4137-89c5-f2e888f0f187 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:31:16.300904 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 680/1344] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:31:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:31:16.308224 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b24907a-0d29-4ff6-a6b5-fbca6c95bb71 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:31:16.310449 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b24907a-0d29-4ff6-a6b5-fbca6c95bb71 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:31:16.310748 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b24907a-0d29-4ff6-a6b5-fbca6c95bb71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:31:16.345015 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-2b24907a-0d29-4ff6-a6b5-fbca6c95bb71 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:31:16.346198 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-2b24907a-0d29-4ff6-a6b5-fbca6c95bb71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:31:16.364106 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 665/1345] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:31:16 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:31:16.407488 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc27a292-b859-4137-89c5-f2e888f0f187 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:31:16.409242 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-dc27a292-b859-4137-89c5-f2e888f0f187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:31:16.423095 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc27a292-b859-4137-89c5-f2e888f0f187 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:31:16.426650 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e63e65-47c7-4667-8043-a152b6fc0029 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:31:16.427299 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e63e65-47c7-4667-8043-a152b6fc0029 service neutron] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:31:16.427589 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e63e65-47c7-4667-8043-a152b6fc0029 service neutron] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:31:16.427861 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e63e65-47c7-4667-8043-a152b6fc0029 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:31:16.437816 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e63e65-47c7-4667-8043-a152b6fc0029 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:31:16.439257 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e63e65-47c7-4667-8043-a152b6fc0029 service neutron] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:31:16.449218 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 681/1346] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:31:16 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:31:21.579737 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbffdd0-cd57-4c2a-ade7-16894bbdf62b None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:31:21.580629 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbffdd0-cd57-4c2a-ade7-16894bbdf62b None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:31:21.580629 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbffdd0-cd57-4c2a-ade7-16894bbdf62b None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:31:21.581718 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 666/1347] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:31:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 14:31:21.589395 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892d40e1-592e-4059-9842-18f23a246c89 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:31:21.589395 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892d40e1-592e-4059-9842-18f23a246c89 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:31:21.589395 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892d40e1-592e-4059-9842-18f23a246c89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:31:21.616552 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-892d40e1-592e-4059-9842-18f23a246c89 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:31:21.617939 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-892d40e1-592e-4059-9842-18f23a246c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:31:21.628598 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 682/1348] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:31:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:31:21.668530 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbffdd0-cd57-4c2a-ade7-16894bbdf62b None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:31:21.679753 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbffdd0-cd57-4c2a-ade7-16894bbdf62b None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:31:21.683326 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fc4bab-29a1-4d69-adb7-77ba3f858bd7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:31:21.683976 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fc4bab-29a1-4d69-adb7-77ba3f858bd7 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:31:21.684257 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fc4bab-29a1-4d69-adb7-77ba3f858bd7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:31:21.684565 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fc4bab-29a1-4d69-adb7-77ba3f858bd7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:31:21.690741 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:31:21.695300 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fc4bab-29a1-4d69-adb7-77ba3f858bd7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:31:21.696895 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fc4bab-29a1-4d69-adb7-77ba3f858bd7 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:31:21.708319 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 667/1349] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:31:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:31:27.335040 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f99ffef-b3dc-4379-a9b0-c15dca4be6aa None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:31:27.339714 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f99ffef-b3dc-4379-a9b0-c15dca4be6aa None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:31:27.340262 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f99ffef-b3dc-4379-a9b0-c15dca4be6aa None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:31:27.344187 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 683/1350] 10.222.0.85 () {54 vars in 921 bytes} [Tue Jun 7 14:31:27 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:31:27.353580 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e67494-2b72-4b84-aede-82755fe50210 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:31:27.355022 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e67494-2b72-4b84-aede-82755fe50210 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:31:27.356068 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e67494-2b72-4b84-aede-82755fe50210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:31:27.492928 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-32e67494-2b72-4b84-aede-82755fe50210 None None] MFA Rules not processed for user `6330294b1c8e4dcd85aad189f007b0aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:31:27.494584 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-32e67494-2b72-4b84-aede-82755fe50210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:31:27.585831 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 668/1351] 10.222.0.85 () {58 vars in 1018 bytes} [Tue Jun 7 14:31:27 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:31:31.607690 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=83 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:31:32.497707 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15ef26e-c71a-4288-a760-2f6b32a35114 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:31:32.498424 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15ef26e-c71a-4288-a760-2f6b32a35114 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:31:32.498815 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15ef26e-c71a-4288-a760-2f6b32a35114 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:31:32.500331 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 684/1352] 10.222.0.85 () {54 vars in 921 bytes} [Tue Jun 7 14:31:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:31:32.505935 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d93beee-ea1c-493a-983b-7ec04f37a54c None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:31:32.506471 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d93beee-ea1c-493a-983b-7ec04f37a54c None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:31:32.506952 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d93beee-ea1c-493a-983b-7ec04f37a54c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:31:32.530999 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-1d93beee-ea1c-493a-983b-7ec04f37a54c None None] MFA Rules not processed for user `6330294b1c8e4dcd85aad189f007b0aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:31:32.532249 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-1d93beee-ea1c-493a-983b-7ec04f37a54c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:31:32.545307 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 669/1353] 10.222.0.85 () {58 vars in 1018 bytes} [Tue Jun 7 14:31:32 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:31:34.148087 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbd5017-3776-445e-bf64-bfa455bf25ed None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:31:34.149076 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbd5017-3776-445e-bf64-bfa455bf25ed None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:31:34.149076 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbd5017-3776-445e-bf64-bfa455bf25ed None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:31:34.150050 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 685/1354] 10.222.0.85 () {54 vars in 921 bytes} [Tue Jun 7 14:31:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:31:34.156820 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60e7c53-57ce-44cf-8f34-817a086946f3 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:31:34.157309 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60e7c53-57ce-44cf-8f34-817a086946f3 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:31:34.157394 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60e7c53-57ce-44cf-8f34-817a086946f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:31:34.178175 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-e60e7c53-57ce-44cf-8f34-817a086946f3 None None] MFA Rules not processed for user `6330294b1c8e4dcd85aad189f007b0aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:31:34.179368 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-e60e7c53-57ce-44cf-8f34-817a086946f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:31:34.188301 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 670/1355] 10.222.0.85 () {58 vars in 1018 bytes} [Tue Jun 7 14:31:34 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:31:38.199164 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0922f9d1-9ad4-4fe0-940e-219fae38cfc3 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:31:38.199711 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0922f9d1-9ad4-4fe0-940e-219fae38cfc3 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:31:38.199711 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0922f9d1-9ad4-4fe0-940e-219fae38cfc3 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:31:38.201020 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 686/1356] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:31:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:31:38.206606 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f41f7a-f0ce-4e7b-8318-925a00fd0973 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:31:38.207226 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f41f7a-f0ce-4e7b-8318-925a00fd0973 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:31:38.207226 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f41f7a-f0ce-4e7b-8318-925a00fd0973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:31:38.226269 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-35f41f7a-f0ce-4e7b-8318-925a00fd0973 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:31:38.227450 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-35f41f7a-f0ce-4e7b-8318-925a00fd0973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:31:38.241147 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 671/1357] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:31:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:31:38.296674 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c286f601-2fdb-474b-a8f2-0593c0f24c6a None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:31:38.297797 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c286f601-2fdb-474b-a8f2-0593c0f24c6a None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:31:38.298393 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c286f601-2fdb-474b-a8f2-0593c0f24c6a None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:31:38.301044 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 687/1358] 10.222.0.85 () {54 vars in 961 bytes} [Tue Jun 7 14:31:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:31:38.316882 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0024872c-29a3-4fc4-9bab-72da0813757b None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:31:38.332260 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0024872c-29a3-4fc4-9bab-72da0813757b None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:31:38.332260 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0024872c-29a3-4fc4-9bab-72da0813757b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:31:38.386502 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-0024872c-29a3-4fc4-9bab-72da0813757b None None] MFA Rules not processed for user `8e857db4902e42989d0d040bbc8517c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:31:38.387678 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-0024872c-29a3-4fc4-9bab-72da0813757b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:31:38.442271 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:31:38.483144 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 672/1359] 10.222.0.85 () {58 vars in 1058 bytes} [Tue Jun 7 14:31:38 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:31:38.491685 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c286f601-2fdb-474b-a8f2-0593c0f24c6a None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:31:38.508654 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c286f601-2fdb-474b-a8f2-0593c0f24c6a None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:31:38.515970 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26961a97-2b48-449d-8148-39708d6b6926 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e857db4902e42989d0d040bbc8517c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:31:38.516887 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26961a97-2b48-449d-8148-39708d6b6926 service nova] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:31:38.517528 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26961a97-2b48-449d-8148-39708d6b6926 service nova] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:31:38.518009 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26961a97-2b48-449d-8148-39708d6b6926 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:31:38.528953 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26961a97-2b48-449d-8148-39708d6b6926 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:31:38.530857 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26961a97-2b48-449d-8148-39708d6b6926 service nova] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:31:38.543378 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 688/1360] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:31:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 14:31:41.114194 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d17ea03-e767-4cfb-b7b9-c951f561a3aa None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:31:41.114754 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d17ea03-e767-4cfb-b7b9-c951f561a3aa None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:31:41.114754 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d17ea03-e767-4cfb-b7b9-c951f561a3aa None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:31:41.116177 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 673/1361] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:31:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:31:41.121486 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4e8fee-39ae-431d-af4b-44a8e9403cbc None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:31:41.121974 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4e8fee-39ae-431d-af4b-44a8e9403cbc None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:31:41.122057 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4e8fee-39ae-431d-af4b-44a8e9403cbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:31:41.141186 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-0c4e8fee-39ae-431d-af4b-44a8e9403cbc None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:31:41.142380 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-0c4e8fee-39ae-431d-af4b-44a8e9403cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:31:41.152247 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 689/1362] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:31:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:31:41.199833 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d17ea03-e767-4cfb-b7b9-c951f561a3aa None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:31:41.216678 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d17ea03-e767-4cfb-b7b9-c951f561a3aa None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:31:41.222721 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a926e7f-3d7a-49b2-ab21-2cbfc8e93dc0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e857db4902e42989d0d040bbc8517c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:31:41.223473 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a926e7f-3d7a-49b2-ab21-2cbfc8e93dc0 service nova] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:31:41.223830 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a926e7f-3d7a-49b2-ab21-2cbfc8e93dc0 service nova] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:31:41.224173 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a926e7f-3d7a-49b2-ab21-2cbfc8e93dc0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:31:41.235140 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a926e7f-3d7a-49b2-ab21-2cbfc8e93dc0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:31:41.236844 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a926e7f-3d7a-49b2-ab21-2cbfc8e93dc0 service nova] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:31:41.250862 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 674/1363] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:31:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:31:43.989901 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd130ccf-d124-40d2-ae8e-0305505240d4 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:31:43.990960 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd130ccf-d124-40d2-ae8e-0305505240d4 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:31:43.991400 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd130ccf-d124-40d2-ae8e-0305505240d4 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:31:43.994184 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 690/1364] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:31:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:31:43.999147 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bec0aa1-1d16-4554-b225-ffdb928653e7 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:31:43.999988 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bec0aa1-1d16-4554-b225-ffdb928653e7 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:31:44.000663 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bec0aa1-1d16-4554-b225-ffdb928653e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:31:44.023532 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-1bec0aa1-1d16-4554-b225-ffdb928653e7 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:31:44.024839 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-1bec0aa1-1d16-4554-b225-ffdb928653e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:31:44.033406 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 675/1365] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:31:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:31:44.080927 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded5035a-9f1d-419d-9362-d0f546edbd50 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:31:44.081218 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded5035a-9f1d-419d-9362-d0f546edbd50 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:31:44.081520 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded5035a-9f1d-419d-9362-d0f546edbd50 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:31:44.082937 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 691/1366] 10.222.0.85 () {54 vars in 961 bytes} [Tue Jun 7 14:31:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:31:44.089232 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944cdac3-dac2-4e66-b4dd-c8bb2d33a532 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:31:44.089232 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944cdac3-dac2-4e66-b4dd-c8bb2d33a532 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:31:44.089232 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944cdac3-dac2-4e66-b4dd-c8bb2d33a532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:31:44.107869 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-944cdac3-dac2-4e66-b4dd-c8bb2d33a532 None None] MFA Rules not processed for user `8e857db4902e42989d0d040bbc8517c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:31:44.109082 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-944cdac3-dac2-4e66-b4dd-c8bb2d33a532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:31:44.118809 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 676/1367] 10.222.0.85 () {58 vars in 1058 bytes} [Tue Jun 7 14:31:44 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:31:44.127017 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded5035a-9f1d-419d-9362-d0f546edbd50 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:31:44.141958 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded5035a-9f1d-419d-9362-d0f546edbd50 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:31:44.147310 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf3b394-fb86-4250-8406-01c33e24f238 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e857db4902e42989d0d040bbc8517c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:31:44.148109 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf3b394-fb86-4250-8406-01c33e24f238 service nova] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:31:44.148510 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf3b394-fb86-4250-8406-01c33e24f238 service nova] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:31:44.148737 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf3b394-fb86-4250-8406-01c33e24f238 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:31:44.157824 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf3b394-fb86-4250-8406-01c33e24f238 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:31:44.159390 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf3b394-fb86-4250-8406-01c33e24f238 service nova] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:31:44.170173 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 692/1368] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:31:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:31:46.645912 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50104aab-c892-4042-a357-c06b2d3ce8b7 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:31:46.646534 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50104aab-c892-4042-a357-c06b2d3ce8b7 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:31:46.646534 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50104aab-c892-4042-a357-c06b2d3ce8b7 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:31:46.648043 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 677/1369] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:31:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:31:46.653835 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fe9aef-620a-40b9-bd27-36f6a5eb6156 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:31:46.654350 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fe9aef-620a-40b9-bd27-36f6a5eb6156 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:31:46.654443 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fe9aef-620a-40b9-bd27-36f6a5eb6156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:31:46.673069 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-73fe9aef-620a-40b9-bd27-36f6a5eb6156 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:31:46.674269 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-73fe9aef-620a-40b9-bd27-36f6a5eb6156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:31:46.682460 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 693/1370] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:31:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:31:46.727851 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50104aab-c892-4042-a357-c06b2d3ce8b7 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:31:46.743440 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50104aab-c892-4042-a357-c06b2d3ce8b7 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:31:46.749980 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3045910d-8b94-4e28-9c8a-0de7aa24b69a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e857db4902e42989d0d040bbc8517c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:31:46.752104 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3045910d-8b94-4e28-9c8a-0de7aa24b69a service nova] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:31:46.752104 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3045910d-8b94-4e28-9c8a-0de7aa24b69a service nova] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:31:46.752104 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3045910d-8b94-4e28-9c8a-0de7aa24b69a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:31:46.761763 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3045910d-8b94-4e28-9c8a-0de7aa24b69a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:31:46.763378 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3045910d-8b94-4e28-9c8a-0de7aa24b69a service nova] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:31:46.774746 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 678/1371] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:31:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:31:46.859939 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13daa9f7-e830-4a07-92bc-2ac780bbe99f None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:31:46.860224 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13daa9f7-e830-4a07-92bc-2ac780bbe99f None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:31:46.860552 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13daa9f7-e830-4a07-92bc-2ac780bbe99f None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:31:46.862097 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 694/1372] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:31:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:31:46.867646 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d088cb9-1729-4807-a7bf-aadaab72624e None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:31:46.868049 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d088cb9-1729-4807-a7bf-aadaab72624e None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:31:46.868970 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d088cb9-1729-4807-a7bf-aadaab72624e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:31:46.905831 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-8d088cb9-1729-4807-a7bf-aadaab72624e None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:31:46.905831 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-8d088cb9-1729-4807-a7bf-aadaab72624e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:31:46.905831 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 679/1373] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:31:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:31:47.215318 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1519d1-546c-4fa9-8390-3b563fea34d7 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:31:47.215603 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1519d1-546c-4fa9-8390-3b563fea34d7 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:31:47.215886 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1519d1-546c-4fa9-8390-3b563fea34d7 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:31:47.217597 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 695/1374] 10.222.0.85 () {54 vars in 963 bytes} [Tue Jun 7 14:31:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:31:47.223257 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd3848-038b-48d0-8889-3d0fd33c0144 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:31:47.223546 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd3848-038b-48d0-8889-3d0fd33c0144 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:31:47.223828 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd3848-038b-48d0-8889-3d0fd33c0144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:31:47.284197 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-acbd3848-038b-48d0-8889-3d0fd33c0144 None None] MFA Rules not processed for user `89d81f597cb44c56a966537089bcce4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:31:47.285948 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-acbd3848-038b-48d0-8889-3d0fd33c0144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:31:47.373812 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 680/1375] 10.222.0.85 () {58 vars in 1060 bytes} [Tue Jun 7 14:31:47 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:31:47.381477 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1519d1-546c-4fa9-8390-3b563fea34d7 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:31:47.392888 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1519d1-546c-4fa9-8390-3b563fea34d7 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:31:47.396962 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90808cf-826e-4bcb-a89b-127533eb4872 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d81f597cb44c56a966537089bcce4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:31:47.397631 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90808cf-826e-4bcb-a89b-127533eb4872 service cinder] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:31:47.397915 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90808cf-826e-4bcb-a89b-127533eb4872 service cinder] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:31:47.398190 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90808cf-826e-4bcb-a89b-127533eb4872 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:31:47.408695 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90808cf-826e-4bcb-a89b-127533eb4872 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:31:47.410157 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90808cf-826e-4bcb-a89b-127533eb4872 service cinder] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:31:47.422140 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 696/1376] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:31:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:31:49.310197 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c787fee6-da8d-48c0-956b-6868edcfc2ea None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:31:49.310728 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c787fee6-da8d-48c0-956b-6868edcfc2ea None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:31:49.310728 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c787fee6-da8d-48c0-956b-6868edcfc2ea None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:31:49.312113 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 681/1377] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:31:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:31:49.317344 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4941aa8-2846-47c2-91fc-64edb87a6b94 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:31:49.317739 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4941aa8-2846-47c2-91fc-64edb87a6b94 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:31:49.317909 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4941aa8-2846-47c2-91fc-64edb87a6b94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:31:49.337635 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-c4941aa8-2846-47c2-91fc-64edb87a6b94 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:31:49.338927 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-c4941aa8-2846-47c2-91fc-64edb87a6b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:31:49.352832 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 697/1378] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:31:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:31:49.393861 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c787fee6-da8d-48c0-956b-6868edcfc2ea None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:31:49.409171 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c787fee6-da8d-48c0-956b-6868edcfc2ea None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:31:49.414981 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2907a610-41ce-4a51-8d5a-76e4e0cc238a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e857db4902e42989d0d040bbc8517c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:31:49.415648 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2907a610-41ce-4a51-8d5a-76e4e0cc238a service nova] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:31:49.415935 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2907a610-41ce-4a51-8d5a-76e4e0cc238a service nova] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:31:49.416220 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2907a610-41ce-4a51-8d5a-76e4e0cc238a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:31:49.426061 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2907a610-41ce-4a51-8d5a-76e4e0cc238a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:31:49.427666 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2907a610-41ce-4a51-8d5a-76e4e0cc238a service nova] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:31:49.439148 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 682/1379] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:31:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:31:51.915594 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edabf954-b114-4096-8bbb-d78fad6790de None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:31:51.916219 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edabf954-b114-4096-8bbb-d78fad6790de None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:31:51.916219 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edabf954-b114-4096-8bbb-d78fad6790de None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:31:51.919466 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 698/1380] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:31:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:31:51.926087 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8278fab-7743-4c63-b691-4670bb018074 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:31:51.926558 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8278fab-7743-4c63-b691-4670bb018074 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:31:51.926667 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8278fab-7743-4c63-b691-4670bb018074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:31:51.949989 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-f8278fab-7743-4c63-b691-4670bb018074 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:31:51.951507 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-f8278fab-7743-4c63-b691-4670bb018074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:31:51.964421 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 683/1381] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:31:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:31:52.008418 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edabf954-b114-4096-8bbb-d78fad6790de None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:31:52.023999 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edabf954-b114-4096-8bbb-d78fad6790de None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:31:52.030309 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77de8f3d-cfa7-448d-b247-717ee1b9621b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e857db4902e42989d0d040bbc8517c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:31:52.030970 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77de8f3d-cfa7-448d-b247-717ee1b9621b service nova] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:31:52.031259 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77de8f3d-cfa7-448d-b247-717ee1b9621b service nova] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:31:52.031545 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77de8f3d-cfa7-448d-b247-717ee1b9621b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:31:52.040749 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77de8f3d-cfa7-448d-b247-717ee1b9621b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:31:52.042484 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77de8f3d-cfa7-448d-b247-717ee1b9621b service nova] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:31:52.053321 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 699/1382] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:31:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:31:54.509063 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7f327c-e3ee-4fdc-ac86-4cd75c7b8644 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:31:54.509618 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7f327c-e3ee-4fdc-ac86-4cd75c7b8644 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:31:54.509618 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7f327c-e3ee-4fdc-ac86-4cd75c7b8644 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:31:54.511027 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 684/1383] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:31:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:31:54.517913 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e3fd54-81a6-454a-960a-bfe8ce0452cf None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:31:54.518437 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e3fd54-81a6-454a-960a-bfe8ce0452cf None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:31:54.518833 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e3fd54-81a6-454a-960a-bfe8ce0452cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:31:54.540076 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-f6e3fd54-81a6-454a-960a-bfe8ce0452cf None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:31:54.542070 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-f6e3fd54-81a6-454a-960a-bfe8ce0452cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:31:54.553241 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 700/1384] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:31:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:31:54.594728 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7f327c-e3ee-4fdc-ac86-4cd75c7b8644 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:31:54.617823 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7f327c-e3ee-4fdc-ac86-4cd75c7b8644 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:31:54.623428 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ea3d4e-224e-44df-8098-06304d5a6009 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e857db4902e42989d0d040bbc8517c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:31:54.624085 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ea3d4e-224e-44df-8098-06304d5a6009 service nova] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:31:54.624365 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ea3d4e-224e-44df-8098-06304d5a6009 service nova] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:31:54.624670 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ea3d4e-224e-44df-8098-06304d5a6009 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:31:54.635406 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ea3d4e-224e-44df-8098-06304d5a6009 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:31:54.637350 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ea3d4e-224e-44df-8098-06304d5a6009 service nova] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:31:54.647975 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 685/1385] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:31:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:31:56.974765 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6baef4-9393-4076-bf6a-262b40994c82 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:31:56.975665 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6baef4-9393-4076-bf6a-262b40994c82 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:31:56.975665 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6baef4-9393-4076-bf6a-262b40994c82 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:31:56.976763 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 701/1386] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:31:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:31:56.984301 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fc3831-79a1-4bb7-bca6-650f1079cd04 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:31:56.984952 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fc3831-79a1-4bb7-bca6-650f1079cd04 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:31:56.985368 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fc3831-79a1-4bb7-bca6-650f1079cd04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:31:57.009623 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-d2fc3831-79a1-4bb7-bca6-650f1079cd04 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:31:57.011780 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-d2fc3831-79a1-4bb7-bca6-650f1079cd04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:31:57.025968 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 686/1387] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:31:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:31:57.076122 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6baef4-9393-4076-bf6a-262b40994c82 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:31:57.091985 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6baef4-9393-4076-bf6a-262b40994c82 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:31:57.097695 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418e49a6-4624-41cc-b0f2-ab4c7d59109b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e857db4902e42989d0d040bbc8517c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:31:57.098368 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418e49a6-4624-41cc-b0f2-ab4c7d59109b service nova] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:31:57.098657 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418e49a6-4624-41cc-b0f2-ab4c7d59109b service nova] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:31:57.098941 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418e49a6-4624-41cc-b0f2-ab4c7d59109b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:31:57.108648 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418e49a6-4624-41cc-b0f2-ab4c7d59109b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:31:57.110250 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418e49a6-4624-41cc-b0f2-ab4c7d59109b service nova] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:31:57.120018 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 702/1388] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:31:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:31:59.667070 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64abd34-7e3b-4ad7-b64f-9f2e38572fcd None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:31:59.667582 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64abd34-7e3b-4ad7-b64f-9f2e38572fcd None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:31:59.667582 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64abd34-7e3b-4ad7-b64f-9f2e38572fcd None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:31:59.669035 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 687/1389] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:31:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:31:59.674644 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa473a1-c459-4693-9c23-712063893920 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:31:59.675091 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa473a1-c459-4693-9c23-712063893920 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:31:59.675264 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa473a1-c459-4693-9c23-712063893920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:31:59.695347 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-baa473a1-c459-4693-9c23-712063893920 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:31:59.696925 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-baa473a1-c459-4693-9c23-712063893920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:31:59.707546 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 703/1390] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:31:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:31:59.754220 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64abd34-7e3b-4ad7-b64f-9f2e38572fcd None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:31:59.772997 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64abd34-7e3b-4ad7-b64f-9f2e38572fcd None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:31:59.779923 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbb4391-6cfd-4a19-81e6-029a72d412ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e857db4902e42989d0d040bbc8517c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:31:59.780667 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbb4391-6cfd-4a19-81e6-029a72d412ed service nova] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:31:59.780953 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbb4391-6cfd-4a19-81e6-029a72d412ed service nova] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:31:59.781243 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbb4391-6cfd-4a19-81e6-029a72d412ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:31:59.791130 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbb4391-6cfd-4a19-81e6-029a72d412ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:31:59.792807 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbb4391-6cfd-4a19-81e6-029a72d412ed service nova] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:31:59.803413 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 688/1391] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:31:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:32:02.305004 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29451e5-15ba-4d0f-9df4-2996e1219a7f None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:02.305568 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29451e5-15ba-4d0f-9df4-2996e1219a7f None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:02.305568 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29451e5-15ba-4d0f-9df4-2996e1219a7f None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:02.306911 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 704/1392] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:32:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:32:02.312427 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261be7ac-4495-4cc0-910f-4da99c80309e None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:02.312886 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261be7ac-4495-4cc0-910f-4da99c80309e None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:02.312994 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261be7ac-4495-4cc0-910f-4da99c80309e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:02.330924 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-261be7ac-4495-4cc0-910f-4da99c80309e None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:32:02.332115 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-261be7ac-4495-4cc0-910f-4da99c80309e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:32:02.341007 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 689/1393] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:32:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:32:02.387012 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29451e5-15ba-4d0f-9df4-2996e1219a7f None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:32:02.395365 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=46 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:32:02.405352 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29451e5-15ba-4d0f-9df4-2996e1219a7f None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:32:02.410966 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e70f83-58f8-410a-bfd1-474a9832febe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e857db4902e42989d0d040bbc8517c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:32:02.411609 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e70f83-58f8-410a-bfd1-474a9832febe service nova] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:02.411891 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e70f83-58f8-410a-bfd1-474a9832febe service nova] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:02.412192 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e70f83-58f8-410a-bfd1-474a9832febe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:02.421518 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e70f83-58f8-410a-bfd1-474a9832febe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:32:02.423134 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e70f83-58f8-410a-bfd1-474a9832febe service nova] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:32:02.433356 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 705/1394] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:32:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:32:03.804556 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99753e19-c80d-433c-8304-dfcc2255d500 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:03.805106 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99753e19-c80d-433c-8304-dfcc2255d500 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:03.805106 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99753e19-c80d-433c-8304-dfcc2255d500 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:03.806457 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 690/1395] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:32:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:32:03.811743 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c361a146-a170-4782-b0ba-790e9dc3474f None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:03.812210 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c361a146-a170-4782-b0ba-790e9dc3474f None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:03.812315 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c361a146-a170-4782-b0ba-790e9dc3474f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:03.830338 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-c361a146-a170-4782-b0ba-790e9dc3474f None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:32:03.831517 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-c361a146-a170-4782-b0ba-790e9dc3474f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:32:03.840139 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 706/1396] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:32:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:32:03.897779 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1dcab5-b8ed-4eb9-b5aa-18292947c577 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:03.898063 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1dcab5-b8ed-4eb9-b5aa-18292947c577 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:03.898338 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1dcab5-b8ed-4eb9-b5aa-18292947c577 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:03.899928 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 691/1397] 10.222.0.85 () {54 vars in 962 bytes} [Tue Jun 7 14:32:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:32:03.905868 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d58815-cca6-4794-8cee-1062ca0a6b19 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:03.906494 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d58815-cca6-4794-8cee-1062ca0a6b19 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:03.906494 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d58815-cca6-4794-8cee-1062ca0a6b19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:03.982586 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-e9d58815-cca6-4794-8cee-1062ca0a6b19 None None] MFA Rules not processed for user `b51da9867d9147f198b8077e953c4f9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:32:03.983792 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-e9d58815-cca6-4794-8cee-1062ca0a6b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:32:04.046388 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 707/1398] 10.222.0.85 () {58 vars in 1059 bytes} [Tue Jun 7 14:32:03 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:32:04.054016 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1dcab5-b8ed-4eb9-b5aa-18292947c577 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:32:04.059618 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:32:04.065408 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1dcab5-b8ed-4eb9-b5aa-18292947c577 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:32:04.072912 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7e54cb-5c47-4a33-bf03-acd8a70e29a2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51da9867d9147f198b8077e953c4f9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:32:04.072912 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7e54cb-5c47-4a33-bf03-acd8a70e29a2 service glance] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:04.072912 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7e54cb-5c47-4a33-bf03-acd8a70e29a2 service glance] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:04.072912 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7e54cb-5c47-4a33-bf03-acd8a70e29a2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:04.079959 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7e54cb-5c47-4a33-bf03-acd8a70e29a2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:32:04.081463 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7e54cb-5c47-4a33-bf03-acd8a70e29a2 service glance] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:32:04.091533 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 692/1399] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:32:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:32:04.790400 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f77a78-5436-4287-92be-fe15b72aa21d None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:04.790725 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f77a78-5436-4287-92be-fe15b72aa21d None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:04.791129 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f77a78-5436-4287-92be-fe15b72aa21d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:04.840751 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-e2f77a78-5436-4287-92be-fe15b72aa21d None None] MFA Rules not processed for user `ad9d021b171244d796c320aa9f8450a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:32:04.841979 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-e2f77a78-5436-4287-92be-fe15b72aa21d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:32:04.908363 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 708/1400] 10.222.0.85 () {58 vars in 1009 bytes} [Tue Jun 7 14:32:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:32:04.947721 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d87154-1dc4-4740-ba22-880844a3adde None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:04.948188 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d87154-1dc4-4740-ba22-880844a3adde None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:04.948280 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d87154-1dc4-4740-ba22-880844a3adde None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:04.949687 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 693/1401] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:32:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:32:04.953826 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fbe5e8-564a-4c1a-ab5d-784456d0b13b None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:04.954338 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fbe5e8-564a-4c1a-ab5d-784456d0b13b None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:04.958076 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fbe5e8-564a-4c1a-ab5d-784456d0b13b None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:04.958555 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a222924-a0d5-4651-b929-6d28c166dc0f None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:04.958555 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a222924-a0d5-4651-b929-6d28c166dc0f None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:04.958555 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a222924-a0d5-4651-b929-6d28c166dc0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:04.958861 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 709/1402] 10.222.0.85 () {54 vars in 942 bytes} [Tue Jun 7 14:32:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:32:04.963641 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6769b8-fe2b-435e-ba28-becd3068d11b None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:04.963959 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6769b8-fe2b-435e-ba28-becd3068d11b None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:04.964238 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6769b8-fe2b-435e-ba28-becd3068d11b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:04.974513 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-6a222924-a0d5-4651-b929-6d28c166dc0f None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:32:04.975722 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-6a222924-a0d5-4651-b929-6d28c166dc0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:32:04.984470 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 694/1403] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:32:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:32:05.017923 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:32:05.020917 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-2c6769b8-fe2b-435e-ba28-becd3068d11b None None] MFA Rules not processed for user `35aa67025f66459da0c0fc3cbfd45430`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:32:05.022116 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-2c6769b8-fe2b-435e-ba28-becd3068d11b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:32:05.031225 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a222924-a0d5-4651-b929-6d28c166dc0f None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:32:05.043887 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a222924-a0d5-4651-b929-6d28c166dc0f None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:32:05.051224 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b239a3f-3e61-48ed-ad82-c22bdde1cf68 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e857db4902e42989d0d040bbc8517c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:32:05.051889 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b239a3f-3e61-48ed-ad82-c22bdde1cf68 service nova] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:05.052179 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b239a3f-3e61-48ed-ad82-c22bdde1cf68 service nova] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:05.052483 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b239a3f-3e61-48ed-ad82-c22bdde1cf68 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:05.062198 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b239a3f-3e61-48ed-ad82-c22bdde1cf68 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:32:05.063786 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b239a3f-3e61-48ed-ad82-c22bdde1cf68 service nova] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:32:05.074071 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 695/1404] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:32:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 14:32:05.098727 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 710/1405] 10.222.0.85 () {58 vars in 1039 bytes} [Tue Jun 7 14:32:04 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:32:05.108874 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b239a3f-3e61-48ed-ad82-c22bdde1cf68 service nova] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:32:05.120068 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b239a3f-3e61-48ed-ad82-c22bdde1cf68 service nova] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:32:05.125143 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cae0622-db6e-490a-8e07-dea3d2d07690 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35aa67025f66459da0c0fc3cbfd45430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:32:05.125781 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cae0622-db6e-490a-8e07-dea3d2d07690 service swift] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:05.126069 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cae0622-db6e-490a-8e07-dea3d2d07690 service swift] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:05.126352 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cae0622-db6e-490a-8e07-dea3d2d07690 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:05.135096 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cae0622-db6e-490a-8e07-dea3d2d07690 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:32:05.136708 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cae0622-db6e-490a-8e07-dea3d2d07690 service swift] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:32:05.144820 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 696/1406] 10.222.0.85 () {60 vars in 1384 bytes} [Tue Jun 7 14:32:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 14:32:07.616390 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b475e60a-8a39-4c0e-b051-a85da10c82c0 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:07.616967 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b475e60a-8a39-4c0e-b051-a85da10c82c0 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:07.616967 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b475e60a-8a39-4c0e-b051-a85da10c82c0 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:07.618762 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 711/1407] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:32:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:32:07.627053 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d547d9-9163-4b2e-8b98-24736938a620 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:07.627622 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d547d9-9163-4b2e-8b98-24736938a620 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:07.627709 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d547d9-9163-4b2e-8b98-24736938a620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:07.647190 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-01d547d9-9163-4b2e-8b98-24736938a620 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:32:07.648462 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-01d547d9-9163-4b2e-8b98-24736938a620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:32:07.658310 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 697/1408] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:32:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:32:07.713450 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b475e60a-8a39-4c0e-b051-a85da10c82c0 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:32:07.729176 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b475e60a-8a39-4c0e-b051-a85da10c82c0 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:32:07.735334 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af50f44a-7b0e-460a-9244-a00b36ec75dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e857db4902e42989d0d040bbc8517c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:32:07.736180 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af50f44a-7b0e-460a-9244-a00b36ec75dd service nova] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:07.736498 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af50f44a-7b0e-460a-9244-a00b36ec75dd service nova] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:07.736780 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af50f44a-7b0e-460a-9244-a00b36ec75dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:07.747711 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af50f44a-7b0e-460a-9244-a00b36ec75dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:32:07.749287 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af50f44a-7b0e-460a-9244-a00b36ec75dd service nova] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:32:07.761070 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 712/1409] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:32:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:32:17.796907 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=23 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:32:17.831212 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=36 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:32:30.057493 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab9980c-ff64-45b0-93aa-e2bc5f0c3062 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:30.058097 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab9980c-ff64-45b0-93aa-e2bc5f0c3062 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:30.058097 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab9980c-ff64-45b0-93aa-e2bc5f0c3062 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:30.059434 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 698/1410] 10.222.0.85 () {54 vars in 918 bytes} [Tue Jun 7 14:32:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:32:30.064504 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed28d34-a2e9-4884-8db5-364e613f8cae None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:30.064957 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed28d34-a2e9-4884-8db5-364e613f8cae None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:30.065088 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed28d34-a2e9-4884-8db5-364e613f8cae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:30.083871 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-aed28d34-a2e9-4884-8db5-364e613f8cae None None] MFA Rules not processed for user `6330294b1c8e4dcd85aad189f007b0aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:32:30.085099 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-aed28d34-a2e9-4884-8db5-364e613f8cae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:32:30.094283 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 713/1411] 10.222.0.85 () {58 vars in 1015 bytes} [Tue Jun 7 14:32:30 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:32:30.108994 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649d05a8-697c-4933-bcee-12260452cd52 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:30.109277 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649d05a8-697c-4933-bcee-12260452cd52 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:30.109576 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649d05a8-697c-4933-bcee-12260452cd52 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:30.110953 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 699/1412] 10.222.0.85 () {54 vars in 918 bytes} [Tue Jun 7 14:32:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:32:30.115503 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0f56fc-6c6a-44a5-9b0d-75b59a87822e None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:30.115821 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0f56fc-6c6a-44a5-9b0d-75b59a87822e None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:30.116100 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0f56fc-6c6a-44a5-9b0d-75b59a87822e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:30.134358 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-db0f56fc-6c6a-44a5-9b0d-75b59a87822e None None] MFA Rules not processed for user `8e857db4902e42989d0d040bbc8517c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:32:30.135582 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-db0f56fc-6c6a-44a5-9b0d-75b59a87822e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:32:30.145371 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 714/1413] 10.222.0.85 () {58 vars in 1015 bytes} [Tue Jun 7 14:32:30 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:32:32.914328 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19446a2-ef29-444a-ba9f-74dcceb04fa7 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:32.914926 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19446a2-ef29-444a-ba9f-74dcceb04fa7 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:32.914926 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19446a2-ef29-444a-ba9f-74dcceb04fa7 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:32.916220 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 700/1414] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 14:32:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:32:32.922445 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77069f66-688f-4c9d-9787-defd6d1038d6 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:32.922917 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77069f66-688f-4c9d-9787-defd6d1038d6 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:32.923025 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77069f66-688f-4c9d-9787-defd6d1038d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:32.942394 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-77069f66-688f-4c9d-9787-defd6d1038d6 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:32:32.943699 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-77069f66-688f-4c9d-9787-defd6d1038d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:32:32.951764 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 715/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 14:32:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:32:32.996467 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19446a2-ef29-444a-ba9f-74dcceb04fa7 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:32:33.011030 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19446a2-ef29-444a-ba9f-74dcceb04fa7 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:32:33.016451 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40ea64a-4f7c-4695-a7ba-ec5ee12d6f95 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e857db4902e42989d0d040bbc8517c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:32:33.017094 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40ea64a-4f7c-4695-a7ba-ec5ee12d6f95 service nova] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:33.017381 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40ea64a-4f7c-4695-a7ba-ec5ee12d6f95 service nova] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:33.017664 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40ea64a-4f7c-4695-a7ba-ec5ee12d6f95 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:33.026461 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40ea64a-4f7c-4695-a7ba-ec5ee12d6f95 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:32:33.028594 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40ea64a-4f7c-4695-a7ba-ec5ee12d6f95 service nova] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:32:33.038787 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 701/1416] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:32:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:32:34.042865 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b3c748-74f4-49db-875d-00bfb30eff47 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:34.043434 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b3c748-74f4-49db-875d-00bfb30eff47 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:34.043434 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b3c748-74f4-49db-875d-00bfb30eff47 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:34.044880 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 716/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 14:32:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:32:34.050044 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e89a1e7-7990-47ee-86e6-f9a533247849 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:34.050524 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e89a1e7-7990-47ee-86e6-f9a533247849 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:34.050624 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e89a1e7-7990-47ee-86e6-f9a533247849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:34.069386 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-4e89a1e7-7990-47ee-86e6-f9a533247849 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:32:34.070585 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-4e89a1e7-7990-47ee-86e6-f9a533247849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:32:34.079071 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 702/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 14:32:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:32:34.240538 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b3c748-74f4-49db-875d-00bfb30eff47 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:32:34.256431 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b3c748-74f4-49db-875d-00bfb30eff47 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:32:34.263229 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dd93c9-58ca-40d2-8071-398c6275a8f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e857db4902e42989d0d040bbc8517c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:32:34.263883 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dd93c9-58ca-40d2-8071-398c6275a8f0 service nova] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:34.264177 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dd93c9-58ca-40d2-8071-398c6275a8f0 service nova] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:34.264495 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dd93c9-58ca-40d2-8071-398c6275a8f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:34.273347 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dd93c9-58ca-40d2-8071-398c6275a8f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:32:34.274938 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dd93c9-58ca-40d2-8071-398c6275a8f0 service nova] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:32:34.285423 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 717/1419] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:32:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:32:35.312593 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1586b75-babf-40ad-b434-1d1403db46e0 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:35.313134 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1586b75-babf-40ad-b434-1d1403db46e0 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:35.313134 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1586b75-babf-40ad-b434-1d1403db46e0 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:35.314487 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 703/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 14:32:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:32:35.320967 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb082c6-46da-407e-a1e2-02b8b0879b21 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:35.321447 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb082c6-46da-407e-a1e2-02b8b0879b21 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:35.321562 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb082c6-46da-407e-a1e2-02b8b0879b21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:35.339441 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-7fb082c6-46da-407e-a1e2-02b8b0879b21 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:32:35.341183 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-7fb082c6-46da-407e-a1e2-02b8b0879b21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:32:35.349414 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 718/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 14:32:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:32:35.503640 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1586b75-babf-40ad-b434-1d1403db46e0 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:32:35.519485 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1586b75-babf-40ad-b434-1d1403db46e0 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:32:35.531193 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95755dee-a0ae-464e-8116-b64016af28cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e857db4902e42989d0d040bbc8517c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:32:35.531853 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95755dee-a0ae-464e-8116-b64016af28cb service nova] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:35.532143 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95755dee-a0ae-464e-8116-b64016af28cb service nova] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:35.532454 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95755dee-a0ae-464e-8116-b64016af28cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:35.541462 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95755dee-a0ae-464e-8116-b64016af28cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:32:35.543077 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95755dee-a0ae-464e-8116-b64016af28cb service nova] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:32:35.553161 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 704/1422] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:32:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:32:36.488535 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf21041-ba4d-41a8-924d-46e3902b8522 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:36.490030 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf21041-ba4d-41a8-924d-46e3902b8522 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:36.490030 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf21041-ba4d-41a8-924d-46e3902b8522 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:36.490602 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 719/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 14:32:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 14:32:36.495712 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67939b8b-243d-4b4d-ae94-2ce24d4860fa None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:36.496172 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67939b8b-243d-4b4d-ae94-2ce24d4860fa None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:36.496267 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67939b8b-243d-4b4d-ae94-2ce24d4860fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:36.514445 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-67939b8b-243d-4b4d-ae94-2ce24d4860fa None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:32:36.515639 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-67939b8b-243d-4b4d-ae94-2ce24d4860fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:32:36.523703 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 705/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 14:32:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:32:36.673070 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf21041-ba4d-41a8-924d-46e3902b8522 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:32:36.688270 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf21041-ba4d-41a8-924d-46e3902b8522 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:32:36.694006 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7f40b9-f37d-400f-b6fe-362687f54edb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e857db4902e42989d0d040bbc8517c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:32:36.694652 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7f40b9-f37d-400f-b6fe-362687f54edb service nova] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:36.694934 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7f40b9-f37d-400f-b6fe-362687f54edb service nova] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:36.695211 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7f40b9-f37d-400f-b6fe-362687f54edb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:36.704159 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7f40b9-f37d-400f-b6fe-362687f54edb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:32:36.705784 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7f40b9-f37d-400f-b6fe-362687f54edb service nova] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:32:36.716520 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 720/1425] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:32:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:32:37.666360 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c6272-153e-467f-8b95-278bc9f2b246 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:37.666951 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c6272-153e-467f-8b95-278bc9f2b246 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:37.666951 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c6272-153e-467f-8b95-278bc9f2b246 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:37.669006 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 706/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 14:32:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:32:37.673859 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f878fce7-7dee-46a7-b8db-b1bde438c3cd None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:37.674248 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f878fce7-7dee-46a7-b8db-b1bde438c3cd None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:37.674414 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f878fce7-7dee-46a7-b8db-b1bde438c3cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:37.693133 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-f878fce7-7dee-46a7-b8db-b1bde438c3cd None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:32:37.694309 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-f878fce7-7dee-46a7-b8db-b1bde438c3cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:32:37.702027 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 721/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 14:32:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:32:37.852584 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8c6272-153e-467f-8b95-278bc9f2b246 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:32:37.867411 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8c6272-153e-467f-8b95-278bc9f2b246 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:32:37.873416 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c51756-abe4-4f4b-b5ff-42ae19bc84de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e857db4902e42989d0d040bbc8517c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:32:37.874049 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c51756-abe4-4f4b-b5ff-42ae19bc84de service nova] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:37.874334 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c51756-abe4-4f4b-b5ff-42ae19bc84de service nova] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:37.874611 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c51756-abe4-4f4b-b5ff-42ae19bc84de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:37.882988 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c51756-abe4-4f4b-b5ff-42ae19bc84de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:32:37.884575 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c51756-abe4-4f4b-b5ff-42ae19bc84de service nova] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:32:37.894107 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 707/1428] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:32:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:32:38.839860 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f646318c-00ce-4aa1-a7f6-4ec41df1d953 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:38.840436 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f646318c-00ce-4aa1-a7f6-4ec41df1d953 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:38.840436 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f646318c-00ce-4aa1-a7f6-4ec41df1d953 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:38.841870 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 722/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 14:32:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:32:38.846812 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025465fe-f820-4851-8bc6-2169ad39a1d2 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:38.847269 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025465fe-f820-4851-8bc6-2169ad39a1d2 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:38.847376 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025465fe-f820-4851-8bc6-2169ad39a1d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:38.865460 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-025465fe-f820-4851-8bc6-2169ad39a1d2 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:32:38.866656 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-025465fe-f820-4851-8bc6-2169ad39a1d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:32:38.874995 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 708/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 14:32:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:32:39.026779 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f646318c-00ce-4aa1-a7f6-4ec41df1d953 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:32:39.041131 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f646318c-00ce-4aa1-a7f6-4ec41df1d953 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:32:39.046655 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377501a3-1173-4b09-9f21-70a19a08ea19 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e857db4902e42989d0d040bbc8517c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:32:39.047333 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377501a3-1173-4b09-9f21-70a19a08ea19 service nova] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:39.047619 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377501a3-1173-4b09-9f21-70a19a08ea19 service nova] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:39.047904 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377501a3-1173-4b09-9f21-70a19a08ea19 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:39.056290 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377501a3-1173-4b09-9f21-70a19a08ea19 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:32:39.057912 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377501a3-1173-4b09-9f21-70a19a08ea19 service nova] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:32:39.067979 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 723/1431] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:32:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:32:40.029987 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdccc975-9954-498a-aefb-f577d43cc385 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:40.030512 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdccc975-9954-498a-aefb-f577d43cc385 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:40.030641 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdccc975-9954-498a-aefb-f577d43cc385 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:40.031921 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 709/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 14:32:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:32:40.037246 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07706caa-721c-41c8-a19c-b849d70a76f5 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:40.037695 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07706caa-721c-41c8-a19c-b849d70a76f5 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:40.037791 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07706caa-721c-41c8-a19c-b849d70a76f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:40.056187 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-07706caa-721c-41c8-a19c-b849d70a76f5 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:32:40.057416 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-07706caa-721c-41c8-a19c-b849d70a76f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:32:40.066217 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 724/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 14:32:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:32:40.217285 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdccc975-9954-498a-aefb-f577d43cc385 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:32:40.232249 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdccc975-9954-498a-aefb-f577d43cc385 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:32:40.237520 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4720efe4-260d-4f7f-bcf5-6c90f67f40e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e857db4902e42989d0d040bbc8517c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:32:40.238164 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4720efe4-260d-4f7f-bcf5-6c90f67f40e1 service nova] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:40.238450 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4720efe4-260d-4f7f-bcf5-6c90f67f40e1 service nova] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:40.238729 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4720efe4-260d-4f7f-bcf5-6c90f67f40e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:40.246995 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4720efe4-260d-4f7f-bcf5-6c90f67f40e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:32:40.248586 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4720efe4-260d-4f7f-bcf5-6c90f67f40e1 service nova] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:32:40.258136 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 710/1434] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:32:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:32:41.176985 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a44268-ea7e-4304-8672-9116af119ef7 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:41.177521 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a44268-ea7e-4304-8672-9116af119ef7 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:41.177521 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a44268-ea7e-4304-8672-9116af119ef7 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:41.178883 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 725/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 14:32:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:32:41.184455 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8becd158-9c6f-4e05-bcde-ef41873508ff None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:41.184877 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8becd158-9c6f-4e05-bcde-ef41873508ff None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:41.185032 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8becd158-9c6f-4e05-bcde-ef41873508ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:41.203630 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-8becd158-9c6f-4e05-bcde-ef41873508ff None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:32:41.204858 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-8becd158-9c6f-4e05-bcde-ef41873508ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:32:41.213132 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 711/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 14:32:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:32:41.362739 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a44268-ea7e-4304-8672-9116af119ef7 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:32:41.381045 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a44268-ea7e-4304-8672-9116af119ef7 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:32:41.386979 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80d72ca-0ca6-4d77-9fcc-0c5e18cb4550 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e857db4902e42989d0d040bbc8517c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:32:41.387661 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80d72ca-0ca6-4d77-9fcc-0c5e18cb4550 service nova] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:41.387943 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80d72ca-0ca6-4d77-9fcc-0c5e18cb4550 service nova] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:41.388219 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80d72ca-0ca6-4d77-9fcc-0c5e18cb4550 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:41.402436 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80d72ca-0ca6-4d77-9fcc-0c5e18cb4550 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:32:41.404573 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80d72ca-0ca6-4d77-9fcc-0c5e18cb4550 service nova] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:32:41.414803 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 726/1437] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:32:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:32:42.342471 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7216965-612b-4e55-96d9-795e0713ce96 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:42.343002 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7216965-612b-4e55-96d9-795e0713ce96 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:42.343002 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7216965-612b-4e55-96d9-795e0713ce96 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:42.344341 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 712/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 14:32:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:32:42.349258 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0403a297-412e-497b-8864-614283ec7817 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:42.349747 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0403a297-412e-497b-8864-614283ec7817 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:42.349848 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0403a297-412e-497b-8864-614283ec7817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:42.367667 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-0403a297-412e-497b-8864-614283ec7817 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:32:42.368896 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-0403a297-412e-497b-8864-614283ec7817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:32:42.376701 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 727/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 14:32:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:32:42.558020 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7216965-612b-4e55-96d9-795e0713ce96 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:32:42.573578 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7216965-612b-4e55-96d9-795e0713ce96 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:32:42.579581 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122a73a0-5e66-4989-8322-3436bc1402f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e857db4902e42989d0d040bbc8517c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:32:42.580225 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a73a0-5e66-4989-8322-3436bc1402f0 service nova] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:42.580558 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a73a0-5e66-4989-8322-3436bc1402f0 service nova] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:42.580838 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a73a0-5e66-4989-8322-3436bc1402f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:42.589143 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122a73a0-5e66-4989-8322-3436bc1402f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:32:42.590852 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122a73a0-5e66-4989-8322-3436bc1402f0 service nova] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:32:42.600319 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 713/1440] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:32:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:32:44.711534 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c6e32c-bd60-4026-874d-88d1b5fe9b48 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:44.712137 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c6e32c-bd60-4026-874d-88d1b5fe9b48 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:44.712137 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c6e32c-bd60-4026-874d-88d1b5fe9b48 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:44.713634 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 728/1441] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 14:32:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:32:44.719049 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228ba27d-4728-48d6-9080-3cf135aaacea None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:44.719468 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228ba27d-4728-48d6-9080-3cf135aaacea None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:44.719621 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228ba27d-4728-48d6-9080-3cf135aaacea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:44.738180 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-228ba27d-4728-48d6-9080-3cf135aaacea None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:32:44.739380 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-228ba27d-4728-48d6-9080-3cf135aaacea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:32:44.747873 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 714/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 14:32:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:32:44.786360 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c6e32c-bd60-4026-874d-88d1b5fe9b48 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:32:44.797832 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c6e32c-bd60-4026-874d-88d1b5fe9b48 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:32:44.801329 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46b07d3-2964-4889-bd1e-879387a47485 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:32:44.801963 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b07d3-2964-4889-bd1e-879387a47485 service neutron] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:44.802242 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b07d3-2964-4889-bd1e-879387a47485 service neutron] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:44.802516 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b07d3-2964-4889-bd1e-879387a47485 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:44.812245 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46b07d3-2964-4889-bd1e-879387a47485 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:32:44.813690 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46b07d3-2964-4889-bd1e-879387a47485 service neutron] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:32:44.823167 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 729/1443] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:32:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 14:32:47.242793 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9604bd20-3576-439a-9c3a-f14919572b2e None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:47.242793 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9604bd20-3576-439a-9c3a-f14919572b2e None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:47.242793 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9604bd20-3576-439a-9c3a-f14919572b2e None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:47.242793 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 715/1444] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 14:32:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 14:32:47.247011 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4960da85-45f6-4c74-b956-bfd4ed4f64a4 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:47.247445 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4960da85-45f6-4c74-b956-bfd4ed4f64a4 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:47.247568 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4960da85-45f6-4c74-b956-bfd4ed4f64a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:47.266112 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-4960da85-45f6-4c74-b956-bfd4ed4f64a4 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:32:47.267321 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-4960da85-45f6-4c74-b956-bfd4ed4f64a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:32:47.275265 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 730/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 14:32:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:32:47.314807 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9604bd20-3576-439a-9c3a-f14919572b2e None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:32:47.326796 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9604bd20-3576-439a-9c3a-f14919572b2e None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:32:47.330403 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2a38c2-8639-4908-a684-0473e3f34dbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:32:47.331051 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2a38c2-8639-4908-a684-0473e3f34dbf service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:47.331337 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2a38c2-8639-4908-a684-0473e3f34dbf service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:47.331622 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2a38c2-8639-4908-a684-0473e3f34dbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:47.341891 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2a38c2-8639-4908-a684-0473e3f34dbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:32:47.343319 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2a38c2-8639-4908-a684-0473e3f34dbf service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:32:47.353357 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 716/1446] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:32:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:32:50.086643 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8749b34-5e52-4d47-8cc8-29831a210477 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:50.087239 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8749b34-5e52-4d47-8cc8-29831a210477 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:50.087239 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8749b34-5e52-4d47-8cc8-29831a210477 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:50.088627 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 731/1447] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 14:32:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:32:50.094819 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45adcd42-bac2-45e7-9612-4163b772dd2a None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:50.095322 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45adcd42-bac2-45e7-9612-4163b772dd2a None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:50.095426 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45adcd42-bac2-45e7-9612-4163b772dd2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:50.113859 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-45adcd42-bac2-45e7-9612-4163b772dd2a None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:32:50.115054 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-45adcd42-bac2-45e7-9612-4163b772dd2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:32:50.122975 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 717/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 14:32:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:32:50.162167 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8749b34-5e52-4d47-8cc8-29831a210477 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:32:50.173068 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8749b34-5e52-4d47-8cc8-29831a210477 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:32:50.176759 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad07323-6b7c-42d6-b098-7b893c1a2b74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:32:50.177400 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad07323-6b7c-42d6-b098-7b893c1a2b74 service neutron] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:50.177687 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad07323-6b7c-42d6-b098-7b893c1a2b74 service neutron] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:50.177968 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad07323-6b7c-42d6-b098-7b893c1a2b74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:50.187256 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad07323-6b7c-42d6-b098-7b893c1a2b74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:32:50.188727 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad07323-6b7c-42d6-b098-7b893c1a2b74 service neutron] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:32:50.200300 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 732/1449] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:32:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:32:52.587548 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d080a2e5-3b0d-4ed3-92be-4b8cb5343044 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:52.588128 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d080a2e5-3b0d-4ed3-92be-4b8cb5343044 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:52.588128 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d080a2e5-3b0d-4ed3-92be-4b8cb5343044 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:52.589543 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 718/1450] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 14:32:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:32:52.595773 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48113247-6603-4bbf-ad12-889c8a23226f None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:52.596195 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48113247-6603-4bbf-ad12-889c8a23226f None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:52.596471 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48113247-6603-4bbf-ad12-889c8a23226f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:52.614782 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-48113247-6603-4bbf-ad12-889c8a23226f None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:32:52.615965 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-48113247-6603-4bbf-ad12-889c8a23226f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:32:52.624217 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 733/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 14:32:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:32:52.662608 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d080a2e5-3b0d-4ed3-92be-4b8cb5343044 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:32:52.673226 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d080a2e5-3b0d-4ed3-92be-4b8cb5343044 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:32:52.676681 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40285c76-5706-4fef-9dd6-b87b8e8e45e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:32:52.677313 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40285c76-5706-4fef-9dd6-b87b8e8e45e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:52.677595 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40285c76-5706-4fef-9dd6-b87b8e8e45e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:52.677870 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40285c76-5706-4fef-9dd6-b87b8e8e45e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:52.687160 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40285c76-5706-4fef-9dd6-b87b8e8e45e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:32:52.688586 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40285c76-5706-4fef-9dd6-b87b8e8e45e8 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:32:52.697802 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 719/1452] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:32:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:32:54.984454 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dd8dea-d5e2-4cf9-aadd-9198ce40889c None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:54.984952 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dd8dea-d5e2-4cf9-aadd-9198ce40889c None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:54.985079 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dd8dea-d5e2-4cf9-aadd-9198ce40889c None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:54.986369 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 734/1453] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 14:32:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:32:54.991748 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9319ec04-aa80-4038-b583-9c30cd4cdb61 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:54.992171 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9319ec04-aa80-4038-b583-9c30cd4cdb61 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:54.992402 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9319ec04-aa80-4038-b583-9c30cd4cdb61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:55.009991 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-9319ec04-aa80-4038-b583-9c30cd4cdb61 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:32:55.011179 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-9319ec04-aa80-4038-b583-9c30cd4cdb61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:32:55.018980 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 720/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 14:32:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:32:55.056935 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dd8dea-d5e2-4cf9-aadd-9198ce40889c None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:32:55.067677 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dd8dea-d5e2-4cf9-aadd-9198ce40889c None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:32:55.071615 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c262888b-0028-4559-a5f4-270b6fe53b8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:32:55.072255 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c262888b-0028-4559-a5f4-270b6fe53b8a service neutron] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:55.072585 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c262888b-0028-4559-a5f4-270b6fe53b8a service neutron] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:55.072866 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c262888b-0028-4559-a5f4-270b6fe53b8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:55.081823 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c262888b-0028-4559-a5f4-270b6fe53b8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:32:55.083247 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c262888b-0028-4559-a5f4-270b6fe53b8a service neutron] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:32:55.093140 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 735/1455] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:32:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:32:57.378177 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3179093-d7c9-4915-9ba6-ffc7ddf35939 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:57.378702 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3179093-d7c9-4915-9ba6-ffc7ddf35939 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:57.378702 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3179093-d7c9-4915-9ba6-ffc7ddf35939 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:57.380014 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 721/1456] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 14:32:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:32:57.385822 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c45485b-0676-4939-bcc7-19a57e4b4a41 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:57.386232 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c45485b-0676-4939-bcc7-19a57e4b4a41 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:57.386389 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c45485b-0676-4939-bcc7-19a57e4b4a41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:57.404454 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-2c45485b-0676-4939-bcc7-19a57e4b4a41 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:32:57.405651 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-2c45485b-0676-4939-bcc7-19a57e4b4a41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:32:57.413800 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 736/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 14:32:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:32:57.451257 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3179093-d7c9-4915-9ba6-ffc7ddf35939 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:32:57.461631 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3179093-d7c9-4915-9ba6-ffc7ddf35939 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:32:57.465685 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2acc86-a8de-4ecf-aafb-11a5388fa061 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:32:57.466318 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2acc86-a8de-4ecf-aafb-11a5388fa061 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:57.466596 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2acc86-a8de-4ecf-aafb-11a5388fa061 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:57.466863 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2acc86-a8de-4ecf-aafb-11a5388fa061 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:57.475523 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2acc86-a8de-4ecf-aafb-11a5388fa061 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:32:57.476957 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2acc86-a8de-4ecf-aafb-11a5388fa061 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:32:57.486674 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 722/1458] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:32:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:32:59.788037 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9458cdd2-2a24-4ddb-97ae-eabd803f4b29 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:59.788661 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9458cdd2-2a24-4ddb-97ae-eabd803f4b29 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:59.788661 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9458cdd2-2a24-4ddb-97ae-eabd803f4b29 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:59.790068 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 737/1459] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 14:32:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:32:59.795383 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69e7c76-761c-46aa-a425-03584a6276f2 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:59.795825 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69e7c76-761c-46aa-a425-03584a6276f2 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:59.795946 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69e7c76-761c-46aa-a425-03584a6276f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:59.802510 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=70 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:32:59.814395 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-f69e7c76-761c-46aa-a425-03584a6276f2 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:32:59.815874 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-f69e7c76-761c-46aa-a425-03584a6276f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:32:59.823730 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 723/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 14:32:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:32:59.860564 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9458cdd2-2a24-4ddb-97ae-eabd803f4b29 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:32:59.865922 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=73 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:32:59.871692 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9458cdd2-2a24-4ddb-97ae-eabd803f4b29 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:32:59.878185 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82915dd5-06fb-4924-a424-576ad8adb9e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:32:59.878185 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82915dd5-06fb-4924-a424-576ad8adb9e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:32:59.878185 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82915dd5-06fb-4924-a424-576ad8adb9e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:32:59.878185 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82915dd5-06fb-4924-a424-576ad8adb9e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:32:59.885967 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82915dd5-06fb-4924-a424-576ad8adb9e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:32:59.888025 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82915dd5-06fb-4924-a424-576ad8adb9e3 service neutron] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:32:59.898255 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 738/1461] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:32:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:33:02.177565 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58a7cdb-5438-4f13-b205-a673ef0d8ca2 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:33:02.178104 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58a7cdb-5438-4f13-b205-a673ef0d8ca2 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:33:02.178104 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58a7cdb-5438-4f13-b205-a673ef0d8ca2 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:33:02.179486 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 724/1462] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 14:33:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:33:02.184808 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2179342-8092-4c5f-8bdc-1dd7e8def68a None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:33:02.185218 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2179342-8092-4c5f-8bdc-1dd7e8def68a None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:33:02.185377 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2179342-8092-4c5f-8bdc-1dd7e8def68a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:33:02.203307 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-e2179342-8092-4c5f-8bdc-1dd7e8def68a None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:33:02.204538 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-e2179342-8092-4c5f-8bdc-1dd7e8def68a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:33:02.212449 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 739/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 14:33:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:33:02.249386 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58a7cdb-5438-4f13-b205-a673ef0d8ca2 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:33:02.260273 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58a7cdb-5438-4f13-b205-a673ef0d8ca2 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:33:02.264513 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3d2c59-518d-4e21-8487-38f1051a6ef7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:33:02.265156 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3d2c59-518d-4e21-8487-38f1051a6ef7 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:33:02.265441 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3d2c59-518d-4e21-8487-38f1051a6ef7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:33:02.265719 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3d2c59-518d-4e21-8487-38f1051a6ef7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:33:02.274725 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3d2c59-518d-4e21-8487-38f1051a6ef7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:33:02.276122 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3d2c59-518d-4e21-8487-38f1051a6ef7 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:33:02.286190 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 725/1464] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:33:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:33:04.644044 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f064c88-8522-41af-bd20-50d1d31edb14 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:33:04.644925 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f064c88-8522-41af-bd20-50d1d31edb14 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:33:04.644925 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f064c88-8522-41af-bd20-50d1d31edb14 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:33:04.646052 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 740/1465] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 14:33:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:33:04.651480 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1914e385-dec3-43fe-be81-0842dc446ea3 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:33:04.651941 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1914e385-dec3-43fe-be81-0842dc446ea3 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:33:04.652040 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1914e385-dec3-43fe-be81-0842dc446ea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:33:04.671229 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-1914e385-dec3-43fe-be81-0842dc446ea3 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:33:04.672503 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-1914e385-dec3-43fe-be81-0842dc446ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:33:04.680299 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 726/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 14:33:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:33:04.716573 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f064c88-8522-41af-bd20-50d1d31edb14 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:33:04.727056 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f064c88-8522-41af-bd20-50d1d31edb14 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:33:04.731091 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ed402c-fc7f-45f3-8c9f-83fb01445d41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:33:04.731716 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed402c-fc7f-45f3-8c9f-83fb01445d41 service neutron] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:33:04.732001 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed402c-fc7f-45f3-8c9f-83fb01445d41 service neutron] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:33:04.732273 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed402c-fc7f-45f3-8c9f-83fb01445d41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:33:04.740939 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ed402c-fc7f-45f3-8c9f-83fb01445d41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:33:04.742353 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ed402c-fc7f-45f3-8c9f-83fb01445d41 service neutron] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:33:04.751538 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 741/1467] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:33:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:33:07.019210 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510884f1-023e-4f4a-8f15-e92ac3a50566 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:33:07.019776 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510884f1-023e-4f4a-8f15-e92ac3a50566 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:33:07.019776 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510884f1-023e-4f4a-8f15-e92ac3a50566 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:33:07.021169 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 727/1468] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 14:33:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:33:07.026676 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e781f752-b057-40ed-8670-7370d3eea76e None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:33:07.027130 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e781f752-b057-40ed-8670-7370d3eea76e None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:33:07.027235 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e781f752-b057-40ed-8670-7370d3eea76e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:33:07.045089 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-e781f752-b057-40ed-8670-7370d3eea76e None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:33:07.046362 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-e781f752-b057-40ed-8670-7370d3eea76e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:33:07.054107 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 742/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 14:33:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:33:07.091811 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510884f1-023e-4f4a-8f15-e92ac3a50566 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:33:07.102421 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510884f1-023e-4f4a-8f15-e92ac3a50566 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:33:07.105885 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28227f64-bc4e-4dbf-8bc4-c149ca7859b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:33:07.106525 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28227f64-bc4e-4dbf-8bc4-c149ca7859b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:33:07.106809 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28227f64-bc4e-4dbf-8bc4-c149ca7859b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:33:07.107085 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28227f64-bc4e-4dbf-8bc4-c149ca7859b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:33:07.115860 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28227f64-bc4e-4dbf-8bc4-c149ca7859b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:33:07.117304 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28227f64-bc4e-4dbf-8bc4-c149ca7859b4 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:33:07.127049 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 728/1470] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:33:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:33:17.039809 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=17 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:33:17.123004 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=17 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:33:39.456893 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fde7e67-986b-42e1-9366-28071dec6c24 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:33:39.457620 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fde7e67-986b-42e1-9366-28071dec6c24 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:33:39.457620 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fde7e67-986b-42e1-9366-28071dec6c24 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:33:39.458816 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 743/1471] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:33:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 14:33:39.464286 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7167e2-da42-4adf-aa09-69b0628ab1d0 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:33:39.464745 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7167e2-da42-4adf-aa09-69b0628ab1d0 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:33:39.464850 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7167e2-da42-4adf-aa09-69b0628ab1d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:33:39.482728 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-8e7167e2-da42-4adf-aa09-69b0628ab1d0 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:33:39.483910 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-8e7167e2-da42-4adf-aa09-69b0628ab1d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:33:39.491599 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 729/1472] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:33:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:33:39.538889 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317c9905-792d-4f3d-83bf-3bec982ad5d7 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:33:39.542824 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317c9905-792d-4f3d-83bf-3bec982ad5d7 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:33:39.542824 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317c9905-792d-4f3d-83bf-3bec982ad5d7 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:33:39.542824 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 744/1473] 10.222.0.85 () {54 vars in 962 bytes} [Tue Jun 7 14:33:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:33:39.547318 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d64171-77d4-4540-98d1-9a62eca82000 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:33:39.547613 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d64171-77d4-4540-98d1-9a62eca82000 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:33:39.547899 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d64171-77d4-4540-98d1-9a62eca82000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:33:39.566812 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-a4d64171-77d4-4540-98d1-9a62eca82000 None None] MFA Rules not processed for user `b51da9867d9147f198b8077e953c4f9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:33:39.568042 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-a4d64171-77d4-4540-98d1-9a62eca82000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:33:39.574556 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 730/1474] 10.222.0.85 () {58 vars in 1059 bytes} [Tue Jun 7 14:33:39 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:33:39.581864 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317c9905-792d-4f3d-83bf-3bec982ad5d7 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:33:39.592527 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317c9905-792d-4f3d-83bf-3bec982ad5d7 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:33:39.596195 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47efb4c5-cb9d-4c62-8247-27ec1c34d050 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51da9867d9147f198b8077e953c4f9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:33:39.596886 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47efb4c5-cb9d-4c62-8247-27ec1c34d050 service glance] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:33:39.597176 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47efb4c5-cb9d-4c62-8247-27ec1c34d050 service glance] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:33:39.597456 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47efb4c5-cb9d-4c62-8247-27ec1c34d050 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:33:39.607038 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47efb4c5-cb9d-4c62-8247-27ec1c34d050 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:33:39.608495 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47efb4c5-cb9d-4c62-8247-27ec1c34d050 service glance] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:33:39.617883 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 745/1475] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:33:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:33:42.011972 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a980dbac-b80b-4b97-9244-deaf503613ee None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:33:42.012554 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a980dbac-b80b-4b97-9244-deaf503613ee None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:33:42.012554 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a980dbac-b80b-4b97-9244-deaf503613ee None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:33:42.014419 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 731/1476] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:33:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:33:42.019856 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f1b438-5239-4b8c-9d15-fcd317adeaa7 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:33:42.020262 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f1b438-5239-4b8c-9d15-fcd317adeaa7 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:33:42.020459 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f1b438-5239-4b8c-9d15-fcd317adeaa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:33:42.038510 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-40f1b438-5239-4b8c-9d15-fcd317adeaa7 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:33:42.039702 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-40f1b438-5239-4b8c-9d15-fcd317adeaa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:33:42.047538 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 746/1477] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:33:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:33:42.071357 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a980dbac-b80b-4b97-9244-deaf503613ee None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:33:42.084335 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a980dbac-b80b-4b97-9244-deaf503613ee None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:33:42.088887 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71f9ac1-f8d1-4d5a-ae6a-6f9644db6acd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:33:42.089537 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71f9ac1-f8d1-4d5a-ae6a-6f9644db6acd admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:33:42.089824 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71f9ac1-f8d1-4d5a-ae6a-6f9644db6acd admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:33:42.090104 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71f9ac1-f8d1-4d5a-ae6a-6f9644db6acd admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:33:42.090893 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71f9ac1-f8d1-4d5a-ae6a-6f9644db6acd admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:33:42.092672 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: WARNING py.warnings [None req-a71f9ac1-f8d1-4d5a-ae6a-6f9644db6acd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 14:33:42.092672 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: warnings.warn(msg) Jun 07 14:33:42.092672 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: Jun 07 14:33:42.093809 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71f9ac1-f8d1-4d5a-ae6a-6f9644db6acd admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:33:42.113225 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 732/1478] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jun 7 14:33:42 2022] GET /identity/v3/projects => generated 2553 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:33:44.421833 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290a4e20-595d-4a77-a745-f666eaf7c904 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:33:44.422365 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290a4e20-595d-4a77-a745-f666eaf7c904 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:33:44.422365 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290a4e20-595d-4a77-a745-f666eaf7c904 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:33:44.423675 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 747/1479] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:33:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:33:44.428998 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9228a0d6-6be5-447e-b69c-76a457fc7034 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:33:44.429444 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9228a0d6-6be5-447e-b69c-76a457fc7034 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:33:44.429553 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9228a0d6-6be5-447e-b69c-76a457fc7034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:33:44.447473 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-9228a0d6-6be5-447e-b69c-76a457fc7034 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:33:44.448685 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-9228a0d6-6be5-447e-b69c-76a457fc7034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:33:44.456608 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 733/1480] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:33:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:33:44.500998 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290a4e20-595d-4a77-a745-f666eaf7c904 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:33:44.515732 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290a4e20-595d-4a77-a745-f666eaf7c904 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:33:44.521744 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d17061-737b-4563-b5fc-84c7ad65f80f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e857db4902e42989d0d040bbc8517c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:33:44.522383 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d17061-737b-4563-b5fc-84c7ad65f80f service nova] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:33:44.522671 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d17061-737b-4563-b5fc-84c7ad65f80f service nova] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:33:44.522952 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d17061-737b-4563-b5fc-84c7ad65f80f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:33:44.531720 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d17061-737b-4563-b5fc-84c7ad65f80f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:33:44.533338 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d17061-737b-4563-b5fc-84c7ad65f80f service nova] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:33:44.542909 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 748/1481] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:33:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:33:46.979892 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d536e50b-5b85-4a07-856f-52d34244e7c3 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:33:46.980436 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d536e50b-5b85-4a07-856f-52d34244e7c3 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:33:46.980532 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d536e50b-5b85-4a07-856f-52d34244e7c3 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:33:46.982385 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 734/1482] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:33:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 14:33:46.987562 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586e124f-25dd-46fa-848b-280c87bacea4 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:33:46.987960 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586e124f-25dd-46fa-848b-280c87bacea4 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:33:46.988130 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586e124f-25dd-46fa-848b-280c87bacea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:33:47.006245 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-586e124f-25dd-46fa-848b-280c87bacea4 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:33:47.007452 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-586e124f-25dd-46fa-848b-280c87bacea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:33:47.015620 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 749/1483] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:33:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:33:47.052201 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d536e50b-5b85-4a07-856f-52d34244e7c3 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:33:47.063511 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d536e50b-5b85-4a07-856f-52d34244e7c3 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:33:47.067062 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85366aa-c4eb-4eda-85a9-9940381b0adc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:33:47.067697 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85366aa-c4eb-4eda-85a9-9940381b0adc service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:33:47.067978 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85366aa-c4eb-4eda-85a9-9940381b0adc service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:33:47.068253 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85366aa-c4eb-4eda-85a9-9940381b0adc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:33:47.077198 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85366aa-c4eb-4eda-85a9-9940381b0adc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:33:47.078611 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85366aa-c4eb-4eda-85a9-9940381b0adc service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:33:47.088041 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 735/1484] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:33:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:33:49.430497 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead63afc-8a0b-461c-9cf3-e30738f6da25 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:33:49.430497 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead63afc-8a0b-461c-9cf3-e30738f6da25 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:33:49.431954 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead63afc-8a0b-461c-9cf3-e30738f6da25 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:33:49.432202 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 750/1485] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:33:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:33:49.437584 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412a9dc4-4efe-4501-9b3d-fc3ee3020774 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:33:49.438033 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412a9dc4-4efe-4501-9b3d-fc3ee3020774 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:33:49.438159 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412a9dc4-4efe-4501-9b3d-fc3ee3020774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:33:49.456266 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-412a9dc4-4efe-4501-9b3d-fc3ee3020774 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:33:49.457510 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-412a9dc4-4efe-4501-9b3d-fc3ee3020774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:33:49.465512 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 736/1486] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:33:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:33:49.504809 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead63afc-8a0b-461c-9cf3-e30738f6da25 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:33:49.516394 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead63afc-8a0b-461c-9cf3-e30738f6da25 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:33:49.520328 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a37f340-20f8-47bc-9732-0437cfd3f7ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:33:49.521003 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a37f340-20f8-47bc-9732-0437cfd3f7ca service neutron] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:33:49.521291 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a37f340-20f8-47bc-9732-0437cfd3f7ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:33:49.521572 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a37f340-20f8-47bc-9732-0437cfd3f7ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:33:49.531333 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a37f340-20f8-47bc-9732-0437cfd3f7ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:33:49.532827 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a37f340-20f8-47bc-9732-0437cfd3f7ca service neutron] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:33:49.542387 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 751/1487] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:33:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:33:51.891450 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc3f1b3-8247-4646-a202-724e0469342c None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:33:51.891993 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc3f1b3-8247-4646-a202-724e0469342c None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:33:51.891993 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc3f1b3-8247-4646-a202-724e0469342c None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:33:51.893323 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 737/1488] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:33:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:33:51.898505 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e77d3f8-0b50-4d9a-a4a7-735e0d45aaae None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:33:51.898994 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e77d3f8-0b50-4d9a-a4a7-735e0d45aaae None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:33:51.899094 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e77d3f8-0b50-4d9a-a4a7-735e0d45aaae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:33:51.916918 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-2e77d3f8-0b50-4d9a-a4a7-735e0d45aaae None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:33:51.918114 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-2e77d3f8-0b50-4d9a-a4a7-735e0d45aaae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:33:51.925999 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 752/1489] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:33:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:33:51.966763 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc3f1b3-8247-4646-a202-724e0469342c None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:33:51.978554 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc3f1b3-8247-4646-a202-724e0469342c None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:33:51.982160 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8203d0a8-c0d8-4e12-b383-8da0dab802fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:33:51.982870 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8203d0a8-c0d8-4e12-b383-8da0dab802fd service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:33:51.983155 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8203d0a8-c0d8-4e12-b383-8da0dab802fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:33:51.983432 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8203d0a8-c0d8-4e12-b383-8da0dab802fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:33:51.993862 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8203d0a8-c0d8-4e12-b383-8da0dab802fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:33:51.995295 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8203d0a8-c0d8-4e12-b383-8da0dab802fd service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:33:52.005798 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 738/1490] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:33:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:33:54.721899 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2dab1c-6ea6-4b10-a1a9-ee42037e7692 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:33:54.722485 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2dab1c-6ea6-4b10-a1a9-ee42037e7692 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:33:54.722485 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2dab1c-6ea6-4b10-a1a9-ee42037e7692 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:33:54.723880 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 753/1491] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jun 7 14:33:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:33:54.729323 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b70ea65-fec3-4431-8efd-81e3c2d44724 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:33:54.729751 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b70ea65-fec3-4431-8efd-81e3c2d44724 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:33:54.729881 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b70ea65-fec3-4431-8efd-81e3c2d44724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:33:54.747407 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-0b70ea65-fec3-4431-8efd-81e3c2d44724 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:33:54.748681 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-0b70ea65-fec3-4431-8efd-81e3c2d44724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:33:54.756436 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 739/1492] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jun 7 14:33:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:33:54.796202 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2dab1c-6ea6-4b10-a1a9-ee42037e7692 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:33:54.808189 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2dab1c-6ea6-4b10-a1a9-ee42037e7692 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:33:54.812298 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40a9f6c-4f04-4319-a819-2f27fe4b9a08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:33:54.812971 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a9f6c-4f04-4319-a819-2f27fe4b9a08 service neutron] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:33:54.813258 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a9f6c-4f04-4319-a819-2f27fe4b9a08 service neutron] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:33:54.813539 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a9f6c-4f04-4319-a819-2f27fe4b9a08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:33:54.823014 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40a9f6c-4f04-4319-a819-2f27fe4b9a08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:33:54.824465 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40a9f6c-4f04-4319-a819-2f27fe4b9a08 service neutron] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:33:54.834576 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 754/1493] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:33:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:34:04.742200 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=33 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:34:04.830338 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=33 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:35:30.410504 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a9cadb-15f7-47d5-986e-779a1ccea13a None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:35:30.411065 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a9cadb-15f7-47d5-986e-779a1ccea13a None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:35:30.411065 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a9cadb-15f7-47d5-986e-779a1ccea13a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:35:30.428422 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-a5a9cadb-15f7-47d5-986e-779a1ccea13a None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:35:30.429604 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-a5a9cadb-15f7-47d5-986e-779a1ccea13a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:35:30.437391 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 740/1494] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:35:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:35:30.445225 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c43ddb0-abe1-4cbf-b1be-2159ab617ac8 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:35:30.445741 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c43ddb0-abe1-4cbf-b1be-2159ab617ac8 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:35:30.445841 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c43ddb0-abe1-4cbf-b1be-2159ab617ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:35:30.463714 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-1c43ddb0-abe1-4cbf-b1be-2159ab617ac8 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:35:30.464915 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-1c43ddb0-abe1-4cbf-b1be-2159ab617ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:35:30.473159 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 755/1495] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:35:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:35:30.477504 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a9cadb-15f7-47d5-986e-779a1ccea13a None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:35:30.486877 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a9cadb-15f7-47d5-986e-779a1ccea13a None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:35:30.491298 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f960d2a2-210a-4fef-b80d-31ea387d64e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:35:30.491939 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f960d2a2-210a-4fef-b80d-31ea387d64e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:35:30.492220 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f960d2a2-210a-4fef-b80d-31ea387d64e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:35:30.492533 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f960d2a2-210a-4fef-b80d-31ea387d64e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:35:30.493256 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f960d2a2-210a-4fef-b80d-31ea387d64e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:35:30.493562 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f960d2a2-210a-4fef-b80d-31ea387d64e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:35:30.494556 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: WARNING py.warnings [None req-f960d2a2-210a-4fef-b80d-31ea387d64e9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 07 14:35:30.494556 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: warnings.warn(msg) Jun 07 14:35:30.494556 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: Jun 07 14:35:30.495420 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f960d2a2-210a-4fef-b80d-31ea387d64e9 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:35:30.506571 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 741/1496] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jun 7 14:35:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:35:30.516002 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c43ddb0-abe1-4cbf-b1be-2159ab617ac8 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:35:30.524819 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c43ddb0-abe1-4cbf-b1be-2159ab617ac8 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:35:30.529884 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7832b4-cd52-4b72-94e8-2c2e1edceb30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:35:30.530514 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7832b4-cd52-4b72-94e8-2c2e1edceb30 admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:35:30.530796 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7832b4-cd52-4b72-94e8-2c2e1edceb30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:35:30.531068 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7832b4-cd52-4b72-94e8-2c2e1edceb30 admin admin] PATH_INFO: `/v3/projects` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:35:30.531756 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7832b4-cd52-4b72-94e8-2c2e1edceb30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:35:30.533566 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: WARNING py.warnings [None req-7e7832b4-cd52-4b72-94e8-2c2e1edceb30 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 14:35:30.533566 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: warnings.warn(msg) Jun 07 14:35:30.533566 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: Jun 07 14:35:30.534746 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7832b4-cd52-4b72-94e8-2c2e1edceb30 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:35:30.647422 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 756/1497] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:35:30 2022] POST /identity/v3/projects => generated 377 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:35:30.651716 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f960d2a2-210a-4fef-b80d-31ea387d64e9 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:35:30.662345 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f960d2a2-210a-4fef-b80d-31ea387d64e9 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:35:30.717611 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3c9a25-8225-4ae8-99c5-8265a2d3d99e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:35:30.718278 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3c9a25-8225-4ae8-99c5-8265a2d3d99e admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:35:30.718565 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3c9a25-8225-4ae8-99c5-8265a2d3d99e admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:35:30.718844 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3c9a25-8225-4ae8-99c5-8265a2d3d99e admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:35:30.719525 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3c9a25-8225-4ae8-99c5-8265a2d3d99e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:35:30.720563 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: WARNING py.warnings [None req-ed3c9a25-8225-4ae8-99c5-8265a2d3d99e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 14:35:30.720563 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: warnings.warn(msg) Jun 07 14:35:30.720563 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: Jun 07 14:35:30.721754 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3c9a25-8225-4ae8-99c5-8265a2d3d99e admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:35:30.823423 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 742/1498] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:35:30 2022] POST /identity/v3/users => generated 289 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:35:30.828171 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7832b4-cd52-4b72-94e8-2c2e1edceb30 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:35:30.838625 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7832b4-cd52-4b72-94e8-2c2e1edceb30 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:35:30.843314 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78ba02e-33b2-4e1d-8c6c-66ec3ee3e2cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:35:30.844002 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78ba02e-33b2-4e1d-8c6c-66ec3ee3e2cf admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:35:30.844289 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78ba02e-33b2-4e1d-8c6c-66ec3ee3e2cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:35:30.844593 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78ba02e-33b2-4e1d-8c6c-66ec3ee3e2cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:35:30.845321 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78ba02e-33b2-4e1d-8c6c-66ec3ee3e2cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:35:30.846338 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: WARNING py.warnings [None req-d78ba02e-33b2-4e1d-8c6c-66ec3ee3e2cf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 07 14:35:30.846338 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: warnings.warn(msg) Jun 07 14:35:30.846338 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: Jun 07 14:35:30.847237 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78ba02e-33b2-4e1d-8c6c-66ec3ee3e2cf admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:35:30.860123 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 757/1499] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:35:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:35:30.864006 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3c9a25-8225-4ae8-99c5-8265a2d3d99e admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:35:30.874371 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3c9a25-8225-4ae8-99c5-8265a2d3d99e admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:35:30.878932 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188f64f0-9187-439a-b325-47342ecf8d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:35:30.879591 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188f64f0-9187-439a-b325-47342ecf8d70 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:35:30.879880 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188f64f0-9187-439a-b325-47342ecf8d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:35:30.880162 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188f64f0-9187-439a-b325-47342ecf8d70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:35:30.880925 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188f64f0-9187-439a-b325-47342ecf8d70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:35:30.881934 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: WARNING py.warnings [None req-188f64f0-9187-439a-b325-47342ecf8d70 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 07 14:35:30.881934 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: warnings.warn(msg) Jun 07 14:35:30.881934 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: Jun 07 14:35:30.882815 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188f64f0-9187-439a-b325-47342ecf8d70 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:35:30.897667 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 743/1500] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:35:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:35:30.901638 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78ba02e-33b2-4e1d-8c6c-66ec3ee3e2cf admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:35:30.911552 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78ba02e-33b2-4e1d-8c6c-66ec3ee3e2cf admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:35:30.916226 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09a80b5-81a4-4914-8204-70e72ae030f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:35:30.917161 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09a80b5-81a4-4914-8204-70e72ae030f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:35:30.917448 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09a80b5-81a4-4914-8204-70e72ae030f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:35:30.917724 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09a80b5-81a4-4914-8204-70e72ae030f5 admin admin] PATH_INFO: `/v3/projects/d6f85b22e472461b82d17d8dd2131034/users/dce045f3b0ee48c796c6e324d27078b4/roles/3771341b3be641139e752aab690397a4` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:35:30.931997 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09a80b5-81a4-4914-8204-70e72ae030f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6f85b22e472461b82d17d8dd2131034, user_id=dce045f3b0ee48c796c6e324d27078b4, role_id=3771341b3be641139e752aab690397a4)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:35:30.933061 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: WARNING py.warnings [None req-a09a80b5-81a4-4914-8204-70e72ae030f5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 14:35:30.933061 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: warnings.warn(msg) Jun 07 14:35:30.933061 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: Jun 07 14:35:30.935485 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09a80b5-81a4-4914-8204-70e72ae030f5 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:35:31.004839 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 758/1501] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:35:30 2022] PUT /identity/v3/projects/d6f85b22e472461b82d17d8dd2131034/users/dce045f3b0ee48c796c6e324d27078b4/roles/3771341b3be641139e752aab690397a4 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:35:31.017884 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25e127f-36af-45fa-9916-3024fcfe5fb7 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:35:31.018178 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25e127f-36af-45fa-9916-3024fcfe5fb7 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:35:31.018455 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25e127f-36af-45fa-9916-3024fcfe5fb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:35:31.044388 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-c25e127f-36af-45fa-9916-3024fcfe5fb7 None None] MFA Rules not processed for user `dce045f3b0ee48c796c6e324d27078b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:35:31.045676 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-c25e127f-36af-45fa-9916-3024fcfe5fb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:35:31.114974 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 744/1502] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:35:31 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:35:31.128663 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e856850d-1e99-41e5-956c-3a7e645e6d00 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:35:31.128964 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e856850d-1e99-41e5-956c-3a7e645e6d00 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:35:31.129250 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e856850d-1e99-41e5-956c-3a7e645e6d00 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:35:31.130657 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 759/1503] 10.222.0.85 () {54 vars in 963 bytes} [Tue Jun 7 14:35:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 14:35:31.135587 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc118f08-ea6d-4a67-acee-b0c69ec5e123 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:35:31.135875 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc118f08-ea6d-4a67-acee-b0c69ec5e123 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:35:31.136153 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc118f08-ea6d-4a67-acee-b0c69ec5e123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:35:31.154519 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-fc118f08-ea6d-4a67-acee-b0c69ec5e123 None None] MFA Rules not processed for user `89d81f597cb44c56a966537089bcce4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:35:31.155705 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-fc118f08-ea6d-4a67-acee-b0c69ec5e123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:35:31.198146 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:35:31.205218 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 745/1504] 10.222.0.85 () {58 vars in 1060 bytes} [Tue Jun 7 14:35:31 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:35:31.212735 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e856850d-1e99-41e5-956c-3a7e645e6d00 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:35:31.223130 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e856850d-1e99-41e5-956c-3a7e645e6d00 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:35:31.226621 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb81fa1-bbfa-49bf-9b6c-d50f4bb57d05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d81f597cb44c56a966537089bcce4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:35:31.227262 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb81fa1-bbfa-49bf-9b6c-d50f4bb57d05 service cinder] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:35:31.227596 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb81fa1-bbfa-49bf-9b6c-d50f4bb57d05 service cinder] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:35:31.227880 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb81fa1-bbfa-49bf-9b6c-d50f4bb57d05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:35:31.236696 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb81fa1-bbfa-49bf-9b6c-d50f4bb57d05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:35:31.238129 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb81fa1-bbfa-49bf-9b6c-d50f4bb57d05 service cinder] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:35:31.247706 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 760/1505] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:35:31 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:35:31.288485 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b2426e-0159-4974-8436-da8d07b0b1d8 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:35:31.288773 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b2426e-0159-4974-8436-da8d07b0b1d8 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:35:31.289052 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b2426e-0159-4974-8436-da8d07b0b1d8 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:35:31.290316 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 746/1506] 10.222.0.85 () {52 vars in 829 bytes} [Tue Jun 7 14:35:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:35:31.294407 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb81fa1-bbfa-49bf-9b6c-d50f4bb57d05 service cinder] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:35:31.354057 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb81fa1-bbfa-49bf-9b6c-d50f4bb57d05 service cinder] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:35:31.359011 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35891297-79ff-432f-9f5a-40eb4871212c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:35:31.359875 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35891297-79ff-432f-9f5a-40eb4871212c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:35:31.360156 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35891297-79ff-432f-9f5a-40eb4871212c admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:35:31.360482 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35891297-79ff-432f-9f5a-40eb4871212c admin admin] PATH_INFO: `/v3/users/dce045f3b0ee48c796c6e324d27078b4` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:35:31.361657 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35891297-79ff-432f-9f5a-40eb4871212c admin admin] RBAC: Authorizing `identity:delete_user(user_id=dce045f3b0ee48c796c6e324d27078b4)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:35:31.362666 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: WARNING py.warnings [None req-35891297-79ff-432f-9f5a-40eb4871212c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 14:35:31.362666 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: warnings.warn(msg) Jun 07 14:35:31.362666 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: Jun 07 14:35:31.364058 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35891297-79ff-432f-9f5a-40eb4871212c admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:35:31.613589 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-35891297-79ff-432f-9f5a-40eb4871212c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dce045f3b0ee48c796c6e324d27078b4'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:35:31.614142 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-35891297-79ff-432f-9f5a-40eb4871212c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dce045f3b0ee48c796c6e324d27078b4'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:35:31.614927 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-35891297-79ff-432f-9f5a-40eb4871212c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dce045f3b0ee48c796c6e324d27078b4'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:35:31.627361 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-35891297-79ff-432f-9f5a-40eb4871212c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dce045f3b0ee48c796c6e324d27078b4'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:35:31.645781 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-35891297-79ff-432f-9f5a-40eb4871212c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dce045f3b0ee48c796c6e324d27078b4'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:35:31.649834 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:35:31.749300 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 761/1507] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:35:31 2022] DELETE /identity/v3/users/dce045f3b0ee48c796c6e324d27078b4 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:35:31.753716 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953b0e7b-5b3e-4cfb-854f-ed63dc1c4582 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:35:31.754199 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953b0e7b-5b3e-4cfb-854f-ed63dc1c4582 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:35:31.754310 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953b0e7b-5b3e-4cfb-854f-ed63dc1c4582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:35:31.772067 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-953b0e7b-5b3e-4cfb-854f-ed63dc1c4582 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:35:31.773336 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-953b0e7b-5b3e-4cfb-854f-ed63dc1c4582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:35:31.830827 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 747/1508] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:35:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:35:31.840759 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35891297-79ff-432f-9f5a-40eb4871212c admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:35:31.842170 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-35891297-79ff-432f-9f5a-40eb4871212c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:35:31.899122 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35891297-79ff-432f-9f5a-40eb4871212c admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:35:31.902656 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299149b2-67a5-404b-995d-f1816e99d428 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:35:31.903925 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299149b2-67a5-404b-995d-f1816e99d428 service neutron] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:35:31.904208 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299149b2-67a5-404b-995d-f1816e99d428 service neutron] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:35:31.904520 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299149b2-67a5-404b-995d-f1816e99d428 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:35:31.913431 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299149b2-67a5-404b-995d-f1816e99d428 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:35:31.914882 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299149b2-67a5-404b-995d-f1816e99d428 service neutron] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:35:31.924119 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 762/1509] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:35:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:35:32.359643 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953b0e7b-5b3e-4cfb-854f-ed63dc1c4582 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:35:32.361191 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-953b0e7b-5b3e-4cfb-854f-ed63dc1c4582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:35:32.373432 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953b0e7b-5b3e-4cfb-854f-ed63dc1c4582 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:35:32.379447 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e63c7a3-9bba-4d42-a437-52c1cad06678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:35:32.380723 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e63c7a3-9bba-4d42-a437-52c1cad06678 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:35:32.381008 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e63c7a3-9bba-4d42-a437-52c1cad06678 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:35:32.381286 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e63c7a3-9bba-4d42-a437-52c1cad06678 admin admin] PATH_INFO: `/v3/projects/d6f85b22e472461b82d17d8dd2131034` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:35:32.382531 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e63c7a3-9bba-4d42-a437-52c1cad06678 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6f85b22e472461b82d17d8dd2131034)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:35:32.384363 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: WARNING py.warnings [None req-1e63c7a3-9bba-4d42-a437-52c1cad06678 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 14:35:32.384363 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: warnings.warn(msg) Jun 07 14:35:32.384363 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: Jun 07 14:35:32.386293 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e63c7a3-9bba-4d42-a437-52c1cad06678 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:35:32.499131 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-1e63c7a3-9bba-4d42-a437-52c1cad06678 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6f85b22e472461b82d17d8dd2131034', 'request_id': 'req-1e63c7a3-9bba-4d42-a437-52c1cad06678'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:35:32.565491 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-1e63c7a3-9bba-4d42-a437-52c1cad06678 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6f85b22e472461b82d17d8dd2131034', 'request_id': 'req-1e63c7a3-9bba-4d42-a437-52c1cad06678'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:35:32.572205 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-1e63c7a3-9bba-4d42-a437-52c1cad06678 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6f85b22e472461b82d17d8dd2131034', 'request_id': 'req-1e63c7a3-9bba-4d42-a437-52c1cad06678'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:35:32.591509 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-1e63c7a3-9bba-4d42-a437-52c1cad06678 admin admin] The token cache is being invalidate because project d6f85b22e472461b82d17d8dd2131034 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:35:32.591824 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-1e63c7a3-9bba-4d42-a437-52c1cad06678 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:35:32.592512 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-1e63c7a3-9bba-4d42-a437-52c1cad06678 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:35:32.594186 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 748/1510] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:35:32 2022] DELETE /identity/v3/projects/d6f85b22e472461b82d17d8dd2131034 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:35:41.920130 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:35:42.587105 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:37:08.161601 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053dab16-24bf-4f88-8985-70432c29a571 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:08.162187 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053dab16-24bf-4f88-8985-70432c29a571 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:08.162187 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053dab16-24bf-4f88-8985-70432c29a571 None None] PATH_INFO: `/v3` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:08.163469 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 763/1511] 10.222.0.94 () {54 vars in 925 bytes} [Tue Jun 7 14:37:08 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:37:08.170381 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5c6d03-01f2-4f9e-86e2-76c0d5c13780 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:08.170828 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5c6d03-01f2-4f9e-86e2-76c0d5c13780 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:08.170941 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5c6d03-01f2-4f9e-86e2-76c0d5c13780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:08.188480 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-7d5c6d03-01f2-4f9e-86e2-76c0d5c13780 None None] MFA Rules not processed for user `6330294b1c8e4dcd85aad189f007b0aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:37:08.189909 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-7d5c6d03-01f2-4f9e-86e2-76c0d5c13780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:37:08.253915 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 749/1512] 10.222.0.94 () {58 vars in 1015 bytes} [Tue Jun 7 14:37:08 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:37:09.946625 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccd66d6-0e64-4112-9ad9-dffd547a83ba None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:09.947128 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccd66d6-0e64-4112-9ad9-dffd547a83ba None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:09.947128 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccd66d6-0e64-4112-9ad9-dffd547a83ba None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:09.948538 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 764/1513] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 14:37:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:37:09.953438 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b692ddc-343d-4641-8653-075e236ea8dc None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:09.953847 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b692ddc-343d-4641-8653-075e236ea8dc None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:09.954019 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b692ddc-343d-4641-8653-075e236ea8dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:09.973238 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-7b692ddc-343d-4641-8653-075e236ea8dc None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:37:09.974451 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-7b692ddc-343d-4641-8653-075e236ea8dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:37:10.033524 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 750/1514] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 14:37:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:37:10.198923 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccd66d6-0e64-4112-9ad9-dffd547a83ba None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:10.200638 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-bccd66d6-0e64-4112-9ad9-dffd547a83ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:37:10.279978 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccd66d6-0e64-4112-9ad9-dffd547a83ba None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:10.285466 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5322fc9-ffb0-4c5d-ac99-1d718b3e8fa4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e857db4902e42989d0d040bbc8517c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:10.286120 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5322fc9-ffb0-4c5d-ac99-1d718b3e8fa4 service nova] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:10.286405 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5322fc9-ffb0-4c5d-ac99-1d718b3e8fa4 service nova] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:10.286682 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5322fc9-ffb0-4c5d-ac99-1d718b3e8fa4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:10.294959 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5322fc9-ffb0-4c5d-ac99-1d718b3e8fa4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:10.296564 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5322fc9-ffb0-4c5d-ac99-1d718b3e8fa4 service nova] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:10.305704 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 765/1515] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:37:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:37:11.191876 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ad7769-4fc4-480e-981b-03af2cb23f02 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:11.192469 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ad7769-4fc4-480e-981b-03af2cb23f02 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:11.192469 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ad7769-4fc4-480e-981b-03af2cb23f02 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:11.193809 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 751/1516] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 14:37:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:37:11.198711 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55b25e5-e5e2-43cc-b49b-e825553513f1 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:11.199113 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55b25e5-e5e2-43cc-b49b-e825553513f1 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:11.199283 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55b25e5-e5e2-43cc-b49b-e825553513f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:11.217934 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-d55b25e5-e5e2-43cc-b49b-e825553513f1 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:37:11.219607 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-d55b25e5-e5e2-43cc-b49b-e825553513f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:37:11.227702 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 766/1517] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 14:37:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:37:11.377764 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ad7769-4fc4-480e-981b-03af2cb23f02 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:11.392911 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ad7769-4fc4-480e-981b-03af2cb23f02 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:11.398450 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a3f8f5-0c7d-4421-9e53-180f443b2197 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e857db4902e42989d0d040bbc8517c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:11.399089 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a3f8f5-0c7d-4421-9e53-180f443b2197 service nova] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:11.399374 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a3f8f5-0c7d-4421-9e53-180f443b2197 service nova] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:11.399653 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a3f8f5-0c7d-4421-9e53-180f443b2197 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:11.408398 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a3f8f5-0c7d-4421-9e53-180f443b2197 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:11.410007 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a3f8f5-0c7d-4421-9e53-180f443b2197 service nova] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:11.419355 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 752/1518] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:37:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:37:15.442578 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63a28ed-c8e9-4203-bb61-9c9e8916cc79 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:15.443133 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63a28ed-c8e9-4203-bb61-9c9e8916cc79 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:15.443133 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63a28ed-c8e9-4203-bb61-9c9e8916cc79 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:15.444421 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 767/1519] 10.222.0.85 () {54 vars in 960 bytes} [Tue Jun 7 14:37:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:37:15.449459 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f69f1a-855f-4bbc-b2bd-5413502246cb None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:15.450357 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f69f1a-855f-4bbc-b2bd-5413502246cb None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:15.450357 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f69f1a-855f-4bbc-b2bd-5413502246cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:15.467956 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-22f69f1a-855f-4bbc-b2bd-5413502246cb None None] MFA Rules not processed for user `6330294b1c8e4dcd85aad189f007b0aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:37:15.469226 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-22f69f1a-855f-4bbc-b2bd-5413502246cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:37:15.478719 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 753/1520] 10.222.0.85 () {58 vars in 1057 bytes} [Tue Jun 7 14:37:15 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:37:15.486653 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63a28ed-c8e9-4203-bb61-9c9e8916cc79 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:15.499852 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63a28ed-c8e9-4203-bb61-9c9e8916cc79 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:15.504824 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a44ac3-d390-41d1-a5e6-85f6de684c88 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6330294b1c8e4dcd85aad189f007b0aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:15.505457 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a44ac3-d390-41d1-a5e6-85f6de684c88 service placement] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:15.505740 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a44ac3-d390-41d1-a5e6-85f6de684c88 service placement] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:15.506016 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a44ac3-d390-41d1-a5e6-85f6de684c88 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:15.514478 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a44ac3-d390-41d1-a5e6-85f6de684c88 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:15.516050 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a44ac3-d390-41d1-a5e6-85f6de684c88 service placement] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:15.525707 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 768/1521] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:37:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:37:22.709821 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d6bd35-27e4-4b54-a95a-36328971ebcd None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:22.710401 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d6bd35-27e4-4b54-a95a-36328971ebcd None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:22.710401 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d6bd35-27e4-4b54-a95a-36328971ebcd None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:22.711751 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 754/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 14:37:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:37:22.716759 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5397f4-747a-4605-a0bb-d6242241d8fc None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:22.717157 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5397f4-747a-4605-a0bb-d6242241d8fc None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:22.717332 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5397f4-747a-4605-a0bb-d6242241d8fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:22.735743 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-bc5397f4-747a-4605-a0bb-d6242241d8fc None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:37:22.736997 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-bc5397f4-747a-4605-a0bb-d6242241d8fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:37:22.744708 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 769/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 14:37:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:37:22.896739 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d6bd35-27e4-4b54-a95a-36328971ebcd None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:22.911142 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d6bd35-27e4-4b54-a95a-36328971ebcd None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:22.916508 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea3006f-a279-4670-98c7-775a0e848a99 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e857db4902e42989d0d040bbc8517c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:22.917143 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea3006f-a279-4670-98c7-775a0e848a99 service nova] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:22.917426 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea3006f-a279-4670-98c7-775a0e848a99 service nova] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:22.918341 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea3006f-a279-4670-98c7-775a0e848a99 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:22.926359 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea3006f-a279-4670-98c7-775a0e848a99 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:22.927906 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea3006f-a279-4670-98c7-775a0e848a99 service nova] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:22.937316 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 755/1524] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:37:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:37:23.810263 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38f5bff-d8e2-4b60-bff0-5d8fe940ce79 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:23.810800 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38f5bff-d8e2-4b60-bff0-5d8fe940ce79 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:23.810900 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38f5bff-d8e2-4b60-bff0-5d8fe940ce79 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:23.812399 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 770/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 14:37:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:37:23.817360 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb3f4dd-4388-43eb-afed-806495a5232d None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:23.817784 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb3f4dd-4388-43eb-afed-806495a5232d None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:23.817901 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb3f4dd-4388-43eb-afed-806495a5232d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:23.835808 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-9eb3f4dd-4388-43eb-afed-806495a5232d None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:37:23.837075 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-9eb3f4dd-4388-43eb-afed-806495a5232d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:37:23.849498 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 756/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 14:37:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:37:24.000842 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38f5bff-d8e2-4b60-bff0-5d8fe940ce79 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:24.016191 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38f5bff-d8e2-4b60-bff0-5d8fe940ce79 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:24.022559 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d1f86a-4d5c-423c-9035-9f1f8a2ea3b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e857db4902e42989d0d040bbc8517c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:24.023204 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d1f86a-4d5c-423c-9035-9f1f8a2ea3b7 service nova] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:24.023494 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d1f86a-4d5c-423c-9035-9f1f8a2ea3b7 service nova] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:24.023777 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d1f86a-4d5c-423c-9035-9f1f8a2ea3b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:24.032920 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d1f86a-4d5c-423c-9035-9f1f8a2ea3b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:24.034504 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d1f86a-4d5c-423c-9035-9f1f8a2ea3b7 service nova] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:24.043952 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 771/1527] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:37:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:37:28.296446 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8091f139-fdf7-45fb-a4f3-94bb51c3a259 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:28.296986 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8091f139-fdf7-45fb-a4f3-94bb51c3a259 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:28.296986 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8091f139-fdf7-45fb-a4f3-94bb51c3a259 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:28.298288 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 757/1528] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 14:37:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:37:28.303926 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdeadcd-b1ab-4e1c-8b90-ce46bf5a9998 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:28.304430 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdeadcd-b1ab-4e1c-8b90-ce46bf5a9998 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:28.304550 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdeadcd-b1ab-4e1c-8b90-ce46bf5a9998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:28.323672 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-bfdeadcd-b1ab-4e1c-8b90-ce46bf5a9998 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:37:28.324917 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-bfdeadcd-b1ab-4e1c-8b90-ce46bf5a9998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:37:28.333596 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 772/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 14:37:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:37:28.370343 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8091f139-fdf7-45fb-a4f3-94bb51c3a259 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:28.372276 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-8091f139-fdf7-45fb-a4f3-94bb51c3a259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:37:28.431422 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8091f139-fdf7-45fb-a4f3-94bb51c3a259 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:28.434980 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc0f0e1-3066-42ee-930f-1bf5893310b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:28.435629 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc0f0e1-3066-42ee-930f-1bf5893310b7 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:28.435917 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc0f0e1-3066-42ee-930f-1bf5893310b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:28.436198 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc0f0e1-3066-42ee-930f-1bf5893310b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:28.446662 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc0f0e1-3066-42ee-930f-1bf5893310b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:28.448145 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc0f0e1-3066-42ee-930f-1bf5893310b7 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:28.453389 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:37:28.459365 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 758/1530] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:37:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:37:30.724195 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157dbeb2-ce46-45df-9bda-6e95e8f5c0aa None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:30.724805 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157dbeb2-ce46-45df-9bda-6e95e8f5c0aa None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:30.724805 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157dbeb2-ce46-45df-9bda-6e95e8f5c0aa None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:30.726251 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 773/1531] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 14:37:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:37:30.731530 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2151275f-65fc-4b37-b29d-cc83df364d1a None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:30.732012 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2151275f-65fc-4b37-b29d-cc83df364d1a None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:30.732106 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2151275f-65fc-4b37-b29d-cc83df364d1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:30.750503 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-2151275f-65fc-4b37-b29d-cc83df364d1a None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:37:30.752266 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-2151275f-65fc-4b37-b29d-cc83df364d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:37:30.760051 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 759/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 14:37:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:37:30.796460 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157dbeb2-ce46-45df-9bda-6e95e8f5c0aa None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:30.808192 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157dbeb2-ce46-45df-9bda-6e95e8f5c0aa None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:30.811777 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42ac1a2-81b1-4602-8b3c-ff716236248f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:30.812433 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ac1a2-81b1-4602-8b3c-ff716236248f service neutron] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:30.812713 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ac1a2-81b1-4602-8b3c-ff716236248f service neutron] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:30.812994 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ac1a2-81b1-4602-8b3c-ff716236248f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:30.822878 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42ac1a2-81b1-4602-8b3c-ff716236248f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:30.824333 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42ac1a2-81b1-4602-8b3c-ff716236248f service neutron] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:30.834652 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 774/1533] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:37:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:37:34.675489 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96869788-26d5-4afb-b82d-1f1a4b5ec5b8 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:34.676042 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96869788-26d5-4afb-b82d-1f1a4b5ec5b8 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:34.676042 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96869788-26d5-4afb-b82d-1f1a4b5ec5b8 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:34.677439 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 760/1534] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 14:37:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:37:34.682874 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8f03f1-365b-4484-b0bb-763b729a17c5 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:34.683314 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8f03f1-365b-4484-b0bb-763b729a17c5 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:34.683440 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8f03f1-365b-4484-b0bb-763b729a17c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:34.690408 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=62 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:37:34.702548 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-2f8f03f1-365b-4484-b0bb-763b729a17c5 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:37:34.704283 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-2f8f03f1-365b-4484-b0bb-763b729a17c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:37:34.712348 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 775/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 14:37:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:37:35.010087 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96869788-26d5-4afb-b82d-1f1a4b5ec5b8 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:35.012146 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-96869788-26d5-4afb-b82d-1f1a4b5ec5b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:37:35.071083 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96869788-26d5-4afb-b82d-1f1a4b5ec5b8 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:35.074653 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d248ab-44c1-49e3-afb6-a940fbea624d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d81f597cb44c56a966537089bcce4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:35.075337 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d248ab-44c1-49e3-afb6-a940fbea624d service cinder] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:35.075652 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d248ab-44c1-49e3-afb6-a940fbea624d service cinder] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:35.075927 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d248ab-44c1-49e3-afb6-a940fbea624d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:35.085375 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d248ab-44c1-49e3-afb6-a940fbea624d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:35.086800 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d248ab-44c1-49e3-afb6-a940fbea624d service cinder] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:35.097106 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 761/1536] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:37:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:37:37.411868 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13386efc-8800-43d4-8763-d329e1ff46b2 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:37.412452 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13386efc-8800-43d4-8763-d329e1ff46b2 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:37.412452 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13386efc-8800-43d4-8763-d329e1ff46b2 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:37.413751 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 776/1537] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 14:37:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:37:37.419299 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b41274c-c59c-444f-b35a-9edf9512a263 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:37.419709 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b41274c-c59c-444f-b35a-9edf9512a263 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:37.419883 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b41274c-c59c-444f-b35a-9edf9512a263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:37.439399 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-3b41274c-c59c-444f-b35a-9edf9512a263 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:37:37.440630 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-3b41274c-c59c-444f-b35a-9edf9512a263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:37:37.448498 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 762/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 14:37:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:37:37.720822 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13386efc-8800-43d4-8763-d329e1ff46b2 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:37:37.722532 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-13386efc-8800-43d4-8763-d329e1ff46b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:37:37.736544 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13386efc-8800-43d4-8763-d329e1ff46b2 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:37:37.740110 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a4555f-84fe-4ea1-a792-7fee36245c15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d81f597cb44c56a966537089bcce4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:37:37.740790 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a4555f-84fe-4ea1-a792-7fee36245c15 service cinder] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:37:37.741078 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a4555f-84fe-4ea1-a792-7fee36245c15 service cinder] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:37:37.741361 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a4555f-84fe-4ea1-a792-7fee36245c15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:37:37.751091 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a4555f-84fe-4ea1-a792-7fee36245c15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:37:37.752553 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a4555f-84fe-4ea1-a792-7fee36245c15 service cinder] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:37:37.762569 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 777/1539] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:37:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:37:47.433485 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=32 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:37:47.757828 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=8 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:39:09.072776 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac35e220-a2a8-47d4-9886-acb4648de9f3 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:09.073992 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac35e220-a2a8-47d4-9886-acb4648de9f3 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:09.074512 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac35e220-a2a8-47d4-9886-acb4648de9f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:09.077095 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c4da29-f8ad-4a37-9206-aa27d90df845 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:09.077996 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c4da29-f8ad-4a37-9206-aa27d90df845 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:09.078497 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c4da29-f8ad-4a37-9206-aa27d90df845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:09.107910 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-ac35e220-a2a8-47d4-9886-acb4648de9f3 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:09.113379 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-ac35e220-a2a8-47d4-9886-acb4648de9f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:09.117335 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-23c4da29-f8ad-4a37-9206-aa27d90df845 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:09.118921 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-23c4da29-f8ad-4a37-9206-aa27d90df845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:09.128488 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 763/1540] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:39:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:39:09.134388 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 778/1541] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:39:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:39:09.158500 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b522f65d-4258-45a0-8213-5227a77aaa34 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:09.159116 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd21fa2-6df6-4cb0-9585-774887e207a9 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:09.159251 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd21fa2-6df6-4cb0-9585-774887e207a9 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:09.159546 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd21fa2-6df6-4cb0-9585-774887e207a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:09.159980 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b522f65d-4258-45a0-8213-5227a77aaa34 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:09.162084 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b522f65d-4258-45a0-8213-5227a77aaa34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:09.193714 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-7bd21fa2-6df6-4cb0-9585-774887e207a9 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:09.194946 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-7bd21fa2-6df6-4cb0-9585-774887e207a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:09.197163 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-b522f65d-4258-45a0-8213-5227a77aaa34 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:09.198758 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-b522f65d-4258-45a0-8213-5227a77aaa34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:09.225632 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 779/1542] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:39:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:39:09.227733 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 764/1543] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:39:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:39:09.231701 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b522f65d-4258-45a0-8213-5227a77aaa34 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:09.237925 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd21fa2-6df6-4cb0-9585-774887e207a9 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:09.247742 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b522f65d-4258-45a0-8213-5227a77aaa34 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:09.257274 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9255ce21-8426-4837-9272-fb524d4e72d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:09.260114 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9255ce21-8426-4837-9272-fb524d4e72d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:09.260114 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9255ce21-8426-4837-9272-fb524d4e72d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:09.260114 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9255ce21-8426-4837-9272-fb524d4e72d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:09.260407 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd21fa2-6df6-4cb0-9585-774887e207a9 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:09.260658 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9255ce21-8426-4837-9272-fb524d4e72d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:39:09.261079 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9255ce21-8426-4837-9272-fb524d4e72d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:09.262602 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: WARNING py.warnings [None req-9255ce21-8426-4837-9272-fb524d4e72d9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 07 14:39:09.262602 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: warnings.warn(msg) Jun 07 14:39:09.262602 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: Jun 07 14:39:09.263892 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9255ce21-8426-4837-9272-fb524d4e72d9 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:09.272504 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb824091-63a6-4f08-a401-df7bb35a3321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:09.273194 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb824091-63a6-4f08-a401-df7bb35a3321 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:09.273483 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb824091-63a6-4f08-a401-df7bb35a3321 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:09.273764 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb824091-63a6-4f08-a401-df7bb35a3321 admin admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:09.274515 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb824091-63a6-4f08-a401-df7bb35a3321 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:39:09.274828 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb824091-63a6-4f08-a401-df7bb35a3321 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:09.276467 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb824091-63a6-4f08-a401-df7bb35a3321 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:09.295574 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 780/1544] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jun 7 14:39:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:39:09.301285 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 765/1545] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jun 7 14:39:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:39:09.305690 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9255ce21-8426-4837-9272-fb524d4e72d9 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:09.310581 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b321e60-4a99-43bb-83aa-8d236a0ad62f None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:09.310581 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b321e60-4a99-43bb-83aa-8d236a0ad62f None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:09.310887 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b321e60-4a99-43bb-83aa-8d236a0ad62f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:09.329985 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9255ce21-8426-4837-9272-fb524d4e72d9 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:09.349926 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-0b321e60-4a99-43bb-83aa-8d236a0ad62f None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:09.351145 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-0b321e60-4a99-43bb-83aa-8d236a0ad62f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:09.360908 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fca51a-062d-4e71-a76a-771741cc5270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:09.361592 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fca51a-062d-4e71-a76a-771741cc5270 admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:09.361881 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fca51a-062d-4e71-a76a-771741cc5270 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:09.362165 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fca51a-062d-4e71-a76a-771741cc5270 admin admin] PATH_INFO: `/v3/projects` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:09.362871 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fca51a-062d-4e71-a76a-771741cc5270 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:09.364848 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fca51a-062d-4e71-a76a-771741cc5270 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:09.376942 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 766/1546] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:39:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:39:09.379448 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b321e60-4a99-43bb-83aa-8d236a0ad62f None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:09.396867 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b321e60-4a99-43bb-83aa-8d236a0ad62f None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:09.411514 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377a653b-ab27-400a-a921-226fc00ff036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:09.416221 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377a653b-ab27-400a-a921-226fc00ff036 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:09.416221 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377a653b-ab27-400a-a921-226fc00ff036 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:09.416221 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377a653b-ab27-400a-a921-226fc00ff036 admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:09.416221 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377a653b-ab27-400a-a921-226fc00ff036 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:09.416221 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: WARNING py.warnings [None req-377a653b-ab27-400a-a921-226fc00ff036 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 14:39:09.416221 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: warnings.warn(msg) Jun 07 14:39:09.416221 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: Jun 07 14:39:09.416891 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377a653b-ab27-400a-a921-226fc00ff036 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:09.482202 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 781/1547] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:39:09 2022] POST /identity/v3/projects => generated 379 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:39:09.486011 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c7c83b-d7e1-4075-bb41-1690c2742016 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:09.486305 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c7c83b-d7e1-4075-bb41-1690c2742016 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:09.486606 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c7c83b-d7e1-4075-bb41-1690c2742016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:09.505714 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-d2c7c83b-d7e1-4075-bb41-1690c2742016 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:09.506928 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-d2c7c83b-d7e1-4075-bb41-1690c2742016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:09.554557 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 767/1548] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:39:09 2022] POST /identity/v3/projects => generated 375 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:39:09.554557 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e711dda3-1ac9-4bb4-b131-68ece68806d5 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:09.554557 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e711dda3-1ac9-4bb4-b131-68ece68806d5 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:09.554557 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e711dda3-1ac9-4bb4-b131-68ece68806d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:09.575541 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 782/1549] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:39:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:39:09.575541 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3f532f-11a4-464b-b54c-573bf9799361 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:09.584738 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3f532f-11a4-464b-b54c-573bf9799361 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:09.584738 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3f532f-11a4-464b-b54c-573bf9799361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:09.584941 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-e711dda3-1ac9-4bb4-b131-68ece68806d5 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:09.584941 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-e711dda3-1ac9-4bb4-b131-68ece68806d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:09.598503 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-2c3f532f-11a4-464b-b54c-573bf9799361 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:09.599708 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-2c3f532f-11a4-464b-b54c-573bf9799361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:09.649542 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 768/1550] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:39:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:39:09.650937 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c31158-a8d0-49a2-849a-ac77e548a89e None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:09.651234 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c31158-a8d0-49a2-849a-ac77e548a89e None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:09.651535 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c31158-a8d0-49a2-849a-ac77e548a89e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:09.666335 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 783/1551] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:39:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:39:09.667657 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c8200b-31ed-49ca-a263-2bceedfe0d5f None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:09.667957 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c8200b-31ed-49ca-a263-2bceedfe0d5f None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:09.668234 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c8200b-31ed-49ca-a263-2bceedfe0d5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:09.678273 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-69c31158-a8d0-49a2-849a-ac77e548a89e None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:09.679560 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-69c31158-a8d0-49a2-849a-ac77e548a89e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:09.687381 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-d0c8200b-31ed-49ca-a263-2bceedfe0d5f None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:09.688535 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 769/1552] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:39:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:39:09.688700 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-d0c8200b-31ed-49ca-a263-2bceedfe0d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:09.689258 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c31158-a8d0-49a2-849a-ac77e548a89e None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:09.700300 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 784/1553] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:39:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:39:09.701215 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c8200b-31ed-49ca-a263-2bceedfe0d5f None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:09.710140 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c31158-a8d0-49a2-849a-ac77e548a89e None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:09.711919 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c8200b-31ed-49ca-a263-2bceedfe0d5f None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:09.715223 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b495f32-2984-4a37-8896-c76a0f69475c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:09.720097 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b495f32-2984-4a37-8896-c76a0f69475c admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:09.720097 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b495f32-2984-4a37-8896-c76a0f69475c admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:09.720097 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b495f32-2984-4a37-8896-c76a0f69475c admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:09.721059 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babec25a-076a-4470-b744-7b51f8230e9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:09.721167 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b495f32-2984-4a37-8896-c76a0f69475c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:09.721251 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babec25a-076a-4470-b744-7b51f8230e9c admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:09.721251 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babec25a-076a-4470-b744-7b51f8230e9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:09.721251 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babec25a-076a-4470-b744-7b51f8230e9c admin admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:09.721251 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babec25a-076a-4470-b744-7b51f8230e9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:09.721548 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b495f32-2984-4a37-8896-c76a0f69475c admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:09.721628 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: WARNING py.warnings [None req-babec25a-076a-4470-b744-7b51f8230e9c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 14:39:09.721628 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: warnings.warn(msg) Jun 07 14:39:09.721628 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: Jun 07 14:39:09.722024 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babec25a-076a-4470-b744-7b51f8230e9c admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:09.805251 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 770/1554] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:39:09 2022] POST /identity/v3/users => generated 290 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:39:09.806495 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e99f0e-bec9-4159-85a8-aa7132f359dd None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:09.806788 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e99f0e-bec9-4159-85a8-aa7132f359dd None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:09.807068 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e99f0e-bec9-4159-85a8-aa7132f359dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:09.825121 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-14e99f0e-bec9-4159-85a8-aa7132f359dd None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:09.826313 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-14e99f0e-bec9-4159-85a8-aa7132f359dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:09.833624 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 785/1555] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:39:09 2022] POST /identity/v3/users => generated 288 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:39:09.838772 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdda826d-59ca-46fc-b2f0-91918dc7f41f None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:09.838772 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdda826d-59ca-46fc-b2f0-91918dc7f41f None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:09.838772 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdda826d-59ca-46fc-b2f0-91918dc7f41f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:09.839058 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 771/1556] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:39:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:39:09.839058 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e99f0e-bec9-4159-85a8-aa7132f359dd None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:09.849745 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e99f0e-bec9-4159-85a8-aa7132f359dd None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:09.854516 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2b9919-5114-4321-ae22-f60191716dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:09.855207 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2b9919-5114-4321-ae22-f60191716dc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:09.855348 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-fdda826d-59ca-46fc-b2f0-91918dc7f41f None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:09.855500 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2b9919-5114-4321-ae22-f60191716dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:09.855778 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2b9919-5114-4321-ae22-f60191716dc9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:09.856539 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-fdda826d-59ca-46fc-b2f0-91918dc7f41f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:09.856665 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2b9919-5114-4321-ae22-f60191716dc9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:39:09.856850 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2b9919-5114-4321-ae22-f60191716dc9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:09.858472 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2b9919-5114-4321-ae22-f60191716dc9 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:09.865363 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 786/1557] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:39:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:39:09.870203 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd676916-13e4-479e-ab48-b044178fdd4b None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:09.870203 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd676916-13e4-479e-ab48-b044178fdd4b None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:09.870203 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd676916-13e4-479e-ab48-b044178fdd4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:09.879636 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 772/1558] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jun 7 14:39:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:39:09.879636 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3512867c-d60b-48dd-bb51-fe37bd30f771 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:09.879636 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3512867c-d60b-48dd-bb51-fe37bd30f771 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:09.879636 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3512867c-d60b-48dd-bb51-fe37bd30f771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:09.889630 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-cd676916-13e4-479e-ab48-b044178fdd4b None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:09.890837 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-cd676916-13e4-479e-ab48-b044178fdd4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:09.892686 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-3512867c-d60b-48dd-bb51-fe37bd30f771 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:09.894017 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-3512867c-d60b-48dd-bb51-fe37bd30f771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:09.898629 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 787/1559] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:39:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:39:09.902492 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8de761-b5b0-4e9d-8ffc-0d3a91f54a44 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:09.902492 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8de761-b5b0-4e9d-8ffc-0d3a91f54a44 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:09.902492 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8de761-b5b0-4e9d-8ffc-0d3a91f54a44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:09.905207 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 773/1560] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:39:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:39:09.906019 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3512867c-d60b-48dd-bb51-fe37bd30f771 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:09.919529 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3512867c-d60b-48dd-bb51-fe37bd30f771 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:09.927907 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-aa8de761-b5b0-4e9d-8ffc-0d3a91f54a44 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:09.931073 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-aa8de761-b5b0-4e9d-8ffc-0d3a91f54a44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:09.931281 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01917eb5-b231-46e4-9bb0-90a6dfb40437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:09.937633 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01917eb5-b231-46e4-9bb0-90a6dfb40437 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:09.937987 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01917eb5-b231-46e4-9bb0-90a6dfb40437 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:09.938333 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01917eb5-b231-46e4-9bb0-90a6dfb40437 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:09.939606 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01917eb5-b231-46e4-9bb0-90a6dfb40437 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:09.941292 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01917eb5-b231-46e4-9bb0-90a6dfb40437 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:09.943249 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 788/1561] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:39:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:39:09.945629 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8de761-b5b0-4e9d-8ffc-0d3a91f54a44 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:09.952893 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 774/1562] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:39:09 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:39:09.956954 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c5adb9-8d0f-4055-ac7d-8210865b269c None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:09.956954 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c5adb9-8d0f-4055-ac7d-8210865b269c None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:09.956954 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c5adb9-8d0f-4055-ac7d-8210865b269c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:09.959800 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8de761-b5b0-4e9d-8ffc-0d3a91f54a44 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:09.964897 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017b2097-b930-43b3-b385-3fd07bc8d39d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:09.965596 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017b2097-b930-43b3-b385-3fd07bc8d39d admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:09.965883 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017b2097-b930-43b3-b385-3fd07bc8d39d admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:09.966161 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017b2097-b930-43b3-b385-3fd07bc8d39d admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:09.966895 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017b2097-b930-43b3-b385-3fd07bc8d39d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:09.969091 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017b2097-b930-43b3-b385-3fd07bc8d39d admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:09.972072 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:39:09.976204 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-17c5adb9-8d0f-4055-ac7d-8210865b269c None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:09.977146 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-17c5adb9-8d0f-4055-ac7d-8210865b269c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:09.980808 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 789/1563] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:39:09 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:39:09.981078 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017b2097-b930-43b3-b385-3fd07bc8d39d admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:09.988865 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 775/1564] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:39:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:39:09.990016 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c5adb9-8d0f-4055-ac7d-8210865b269c None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:09.991872 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017b2097-b930-43b3-b385-3fd07bc8d39d admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:09.996718 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f6ffec-e3ca-41a8-b431-81b25f2cfb9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:09.997370 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f6ffec-e3ca-41a8-b431-81b25f2cfb9f admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:09.997649 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f6ffec-e3ca-41a8-b431-81b25f2cfb9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:09.997924 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f6ffec-e3ca-41a8-b431-81b25f2cfb9f admin admin] PATH_INFO: `/v3/domains` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:09.998657 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f6ffec-e3ca-41a8-b431-81b25f2cfb9f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:39:09.998963 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f6ffec-e3ca-41a8-b431-81b25f2cfb9f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:10.000320 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c5adb9-8d0f-4055-ac7d-8210865b269c None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:10.000656 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f6ffec-e3ca-41a8-b431-81b25f2cfb9f admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:10.005623 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ead7d37-e4a6-44b2-9d18-5f58fedca63e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:10.006272 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ead7d37-e4a6-44b2-9d18-5f58fedca63e admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:10.006559 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ead7d37-e4a6-44b2-9d18-5f58fedca63e admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:10.006867 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ead7d37-e4a6-44b2-9d18-5f58fedca63e admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:10.007555 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ead7d37-e4a6-44b2-9d18-5f58fedca63e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:10.010330 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ead7d37-e4a6-44b2-9d18-5f58fedca63e admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:10.015523 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 790/1565] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jun 7 14:39:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:39:10.015523 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f6ffec-e3ca-41a8-b431-81b25f2cfb9f admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:10.023931 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f6ffec-e3ca-41a8-b431-81b25f2cfb9f admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:10.028672 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c64e492-dd50-4182-83df-c7a32a417464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:10.029315 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c64e492-dd50-4182-83df-c7a32a417464 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:10.029598 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c64e492-dd50-4182-83df-c7a32a417464 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:10.029874 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c64e492-dd50-4182-83df-c7a32a417464 admin admin] PATH_INFO: `/v3/domains` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:10.030628 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c64e492-dd50-4182-83df-c7a32a417464 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:39:10.030939 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c64e492-dd50-4182-83df-c7a32a417464 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:10.033204 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c64e492-dd50-4182-83df-c7a32a417464 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:10.054396 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 791/1566] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jun 7 14:39:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:39:10.055322 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c64e492-dd50-4182-83df-c7a32a417464 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:10.064963 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c64e492-dd50-4182-83df-c7a32a417464 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:10.070703 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980a0d93-4805-45ac-97e4-b7865fb9875d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:10.071689 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980a0d93-4805-45ac-97e4-b7865fb9875d admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:10.072230 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980a0d93-4805-45ac-97e4-b7865fb9875d admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:10.072766 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980a0d93-4805-45ac-97e4-b7865fb9875d admin admin] PATH_INFO: `/v3/domains` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:10.073791 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980a0d93-4805-45ac-97e4-b7865fb9875d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:39:10.074241 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980a0d93-4805-45ac-97e4-b7865fb9875d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:10.076843 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980a0d93-4805-45ac-97e4-b7865fb9875d admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:10.091288 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 776/1567] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:39:09 2022] POST /identity/v3/projects => generated 385 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:39:10.092498 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ead7d37-e4a6-44b2-9d18-5f58fedca63e admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:10.096069 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 792/1568] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jun 7 14:39:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:39:10.097291 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980a0d93-4805-45ac-97e4-b7865fb9875d admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:10.110495 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980a0d93-4805-45ac-97e4-b7865fb9875d admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:10.119784 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:39:10.175209 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ead7d37-e4a6-44b2-9d18-5f58fedca63e admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:10.175459 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eb421c-07bc-4bf2-bca3-dff624de65da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:10.176216 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eb421c-07bc-4bf2-bca3-dff624de65da admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:10.176539 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eb421c-07bc-4bf2-bca3-dff624de65da admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:10.176942 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eb421c-07bc-4bf2-bca3-dff624de65da admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:10.177572 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eb421c-07bc-4bf2-bca3-dff624de65da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:10.179177 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eb421c-07bc-4bf2-bca3-dff624de65da admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:10.180540 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42fef03-c390-4b73-9700-dafc08abdd8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:10.181472 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42fef03-c390-4b73-9700-dafc08abdd8c admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:10.181602 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42fef03-c390-4b73-9700-dafc08abdd8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:10.181781 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42fef03-c390-4b73-9700-dafc08abdd8c admin admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:10.182679 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42fef03-c390-4b73-9700-dafc08abdd8c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:39:10.182994 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42fef03-c390-4b73-9700-dafc08abdd8c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:10.184682 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42fef03-c390-4b73-9700-dafc08abdd8c admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:10.189485 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 793/1569] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:39:10 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:39:10.190152 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eb421c-07bc-4bf2-bca3-dff624de65da admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:10.199970 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eb421c-07bc-4bf2-bca3-dff624de65da admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:10.200696 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 777/1570] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jun 7 14:39:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:39:10.201672 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42fef03-c390-4b73-9700-dafc08abdd8c admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:10.206489 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e50cc0e-817a-4c95-8aaf-f3f7ee342b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:10.214675 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e50cc0e-817a-4c95-8aaf-f3f7ee342b87 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:10.214675 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e50cc0e-817a-4c95-8aaf-f3f7ee342b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:10.214675 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e50cc0e-817a-4c95-8aaf-f3f7ee342b87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:10.214675 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e50cc0e-817a-4c95-8aaf-f3f7ee342b87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:10.214675 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e50cc0e-817a-4c95-8aaf-f3f7ee342b87 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:10.215094 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42fef03-c390-4b73-9700-dafc08abdd8c admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:10.219231 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c7564e-b09e-4346-bb0b-d0e8ddae04f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:10.219929 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c7564e-b09e-4346-bb0b-d0e8ddae04f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:10.227729 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c7564e-b09e-4346-bb0b-d0e8ddae04f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:10.227729 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c7564e-b09e-4346-bb0b-d0e8ddae04f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:10.227729 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c7564e-b09e-4346-bb0b-d0e8ddae04f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:39:10.228012 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 794/1571] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:39:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:39:10.228119 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c7564e-b09e-4346-bb0b-d0e8ddae04f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:10.228207 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e50cc0e-817a-4c95-8aaf-f3f7ee342b87 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:10.228296 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c7564e-b09e-4346-bb0b-d0e8ddae04f5 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:10.233090 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e50cc0e-817a-4c95-8aaf-f3f7ee342b87 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:10.238491 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 778/1572] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jun 7 14:39:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:39:10.238491 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c7564e-b09e-4346-bb0b-d0e8ddae04f5 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:10.243538 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7b043d-b1f6-490f-a32d-40388ce7cfe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:10.243538 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7b043d-b1f6-490f-a32d-40388ce7cfe8 admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:10.243538 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7b043d-b1f6-490f-a32d-40388ce7cfe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:10.243538 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7b043d-b1f6-490f-a32d-40388ce7cfe8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:10.243851 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7b043d-b1f6-490f-a32d-40388ce7cfe8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:10.244735 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7b043d-b1f6-490f-a32d-40388ce7cfe8 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:10.248047 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c7564e-b09e-4346-bb0b-d0e8ddae04f5 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:10.252860 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e005cf-a8e8-4aa0-b171-03a40c57901f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:10.253578 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e005cf-a8e8-4aa0-b171-03a40c57901f admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:10.253937 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e005cf-a8e8-4aa0-b171-03a40c57901f admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:10.254275 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e005cf-a8e8-4aa0-b171-03a40c57901f admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:10.255026 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e005cf-a8e8-4aa0-b171-03a40c57901f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:10.257020 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e005cf-a8e8-4aa0-b171-03a40c57901f admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:10.354990 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 795/1573] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:39:10 2022] POST /identity/v3/projects => generated 385 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:39:10.355875 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7b043d-b1f6-490f-a32d-40388ce7cfe8 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:10.370278 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 779/1574] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:39:10 2022] POST /identity/v3/projects => generated 367 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:39:10.370278 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e005cf-a8e8-4aa0-b171-03a40c57901f admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:10.436337 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7b043d-b1f6-490f-a32d-40388ce7cfe8 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:10.441026 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9848d8-8208-41b4-8787-037930323399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:10.441732 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9848d8-8208-41b4-8787-037930323399 admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:10.442022 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9848d8-8208-41b4-8787-037930323399 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:10.442301 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9848d8-8208-41b4-8787-037930323399 admin admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:10.442994 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9848d8-8208-41b4-8787-037930323399 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:10.444947 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9848d8-8208-41b4-8787-037930323399 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:10.445244 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e005cf-a8e8-4aa0-b171-03a40c57901f admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:10.453726 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9de25c-820a-4cd2-bc1c-29fcd6961945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:10.455294 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9de25c-820a-4cd2-bc1c-29fcd6961945 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:10.456165 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9de25c-820a-4cd2-bc1c-29fcd6961945 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:10.456976 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9de25c-820a-4cd2-bc1c-29fcd6961945 admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:10.458039 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9de25c-820a-4cd2-bc1c-29fcd6961945 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:10.460017 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9de25c-820a-4cd2-bc1c-29fcd6961945 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:10.524440 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 796/1575] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:39:10 2022] POST /identity/v3/users => generated 293 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:39:10.525509 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9848d8-8208-41b4-8787-037930323399 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:10.536141 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9848d8-8208-41b4-8787-037930323399 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:10.542585 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e755c637-c9dc-4c55-95aa-f5fd92280326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:10.544091 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e755c637-c9dc-4c55-95aa-f5fd92280326 admin admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:10.544335 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e755c637-c9dc-4c55-95aa-f5fd92280326 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:10.544643 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e755c637-c9dc-4c55-95aa-f5fd92280326 admin admin] PATH_INFO: `/v3/projects/1bd7cee3f5704d01ae9df6d300b30004/users/d800b8efe792477bac3ca063920a4a9a/roles/3771341b3be641139e752aab690397a4` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:10.551310 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 780/1576] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:39:10 2022] POST /identity/v3/projects => generated 403 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:39:10.551310 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9de25c-820a-4cd2-bc1c-29fcd6961945 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:10.566638 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:39:10.566638 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e755c637-c9dc-4c55-95aa-f5fd92280326 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bd7cee3f5704d01ae9df6d300b30004, user_id=d800b8efe792477bac3ca063920a4a9a, role_id=3771341b3be641139e752aab690397a4)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:10.566638 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e755c637-c9dc-4c55-95aa-f5fd92280326 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:10.620965 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9de25c-820a-4cd2-bc1c-29fcd6961945 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:10.626169 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40b9742-04ef-4f85-b878-e89012ce0cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:10.626829 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40b9742-04ef-4f85-b878-e89012ce0cd5 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:10.627106 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40b9742-04ef-4f85-b878-e89012ce0cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:10.627381 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40b9742-04ef-4f85-b878-e89012ce0cd5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:10.628069 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40b9742-04ef-4f85-b878-e89012ce0cd5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:10.629976 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40b9742-04ef-4f85-b878-e89012ce0cd5 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:10.633631 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:39:10.646933 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 797/1577] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:39:10 2022] PUT /identity/v3/projects/1bd7cee3f5704d01ae9df6d300b30004/users/d800b8efe792477bac3ca063920a4a9a/roles/3771341b3be641139e752aab690397a4 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:39:10.648628 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e755c637-c9dc-4c55-95aa-f5fd92280326 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:10.713260 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e755c637-c9dc-4c55-95aa-f5fd92280326 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:10.718042 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a96d414-0667-4a3e-b59c-8ed675c3b03f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:10.719469 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a96d414-0667-4a3e-b59c-8ed675c3b03f admin admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:10.719754 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a96d414-0667-4a3e-b59c-8ed675c3b03f admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:10.720033 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a96d414-0667-4a3e-b59c-8ed675c3b03f admin admin] PATH_INFO: `/v3/projects/97ba26b86c7243dab8e6c6b84ed674e8/users/4fb4b0cf5f8b4274b787f327793964f3/roles/3771341b3be641139e752aab690397a4` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:10.744732 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a96d414-0667-4a3e-b59c-8ed675c3b03f admin admin] RBAC: Authorizing `identity:create_grant(project_id=97ba26b86c7243dab8e6c6b84ed674e8, user_id=4fb4b0cf5f8b4274b787f327793964f3, role_id=3771341b3be641139e752aab690397a4)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:10.744936 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 781/1578] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:39:10 2022] POST /identity/v3/projects => generated 393 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:39:10.745829 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40b9742-04ef-4f85-b878-e89012ce0cd5 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:10.748543 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a96d414-0667-4a3e-b59c-8ed675c3b03f admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:10.809250 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40b9742-04ef-4f85-b878-e89012ce0cd5 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:10.814039 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaa7f0e-a0c8-4b8f-a32e-d84d8ae97c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:10.814756 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaa7f0e-a0c8-4b8f-a32e-d84d8ae97c12 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:10.815091 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaa7f0e-a0c8-4b8f-a32e-d84d8ae97c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:10.815681 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaa7f0e-a0c8-4b8f-a32e-d84d8ae97c12 admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:10.815975 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaa7f0e-a0c8-4b8f-a32e-d84d8ae97c12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:10.818030 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaa7f0e-a0c8-4b8f-a32e-d84d8ae97c12 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:10.823151 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 798/1579] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:39:10 2022] PUT /identity/v3/projects/97ba26b86c7243dab8e6c6b84ed674e8/users/4fb4b0cf5f8b4274b787f327793964f3/roles/3771341b3be641139e752aab690397a4 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:39:10.826424 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a96d414-0667-4a3e-b59c-8ed675c3b03f admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:10.891384 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a96d414-0667-4a3e-b59c-8ed675c3b03f admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:10.897449 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e051dfa-3222-46aa-b431-2c3529ed28d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:10.898507 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e051dfa-3222-46aa-b431-2c3529ed28d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:10.898965 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e051dfa-3222-46aa-b431-2c3529ed28d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:10.899300 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 782/1580] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:39:10 2022] POST /identity/v3/projects => generated 365 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:39:10.899437 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e051dfa-3222-46aa-b431-2c3529ed28d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:10.900641 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e051dfa-3222-46aa-b431-2c3529ed28d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:10.900787 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaa7f0e-a0c8-4b8f-a32e-d84d8ae97c12 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:10.904149 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e051dfa-3222-46aa-b431-2c3529ed28d6 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:10.971270 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 799/1581] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:39:10 2022] POST /identity/v3/users => generated 293 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:39:10.976736 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e051dfa-3222-46aa-b431-2c3529ed28d6 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:10.978293 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaa7f0e-a0c8-4b8f-a32e-d84d8ae97c12 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:10.983023 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7557600-8ab0-44ca-9d40-70bbada763a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:10.983709 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7557600-8ab0-44ca-9d40-70bbada763a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:10.983997 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7557600-8ab0-44ca-9d40-70bbada763a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:10.985101 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7557600-8ab0-44ca-9d40-70bbada763a9 admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:10.985252 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e051dfa-3222-46aa-b431-2c3529ed28d6 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:10.985738 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7557600-8ab0-44ca-9d40-70bbada763a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:10.987681 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7557600-8ab0-44ca-9d40-70bbada763a9 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:10.989792 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25071f23-f6f7-49e1-bf39-9bdee79ebbe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:10.990460 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25071f23-f6f7-49e1-bf39-9bdee79ebbe9 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:10.990745 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25071f23-f6f7-49e1-bf39-9bdee79ebbe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:10.991018 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25071f23-f6f7-49e1-bf39-9bdee79ebbe9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:10.991750 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25071f23-f6f7-49e1-bf39-9bdee79ebbe9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:10.993403 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25071f23-f6f7-49e1-bf39-9bdee79ebbe9 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:11.003908 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 800/1582] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:39:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:39:11.004716 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25071f23-f6f7-49e1-bf39-9bdee79ebbe9 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:11.015062 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25071f23-f6f7-49e1-bf39-9bdee79ebbe9 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:11.019815 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2739add-0a0e-4888-ac2f-bd8ebc43f327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:11.022144 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2739add-0a0e-4888-ac2f-bd8ebc43f327 admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:11.022144 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2739add-0a0e-4888-ac2f-bd8ebc43f327 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:11.022144 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2739add-0a0e-4888-ac2f-bd8ebc43f327 admin admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:11.022144 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2739add-0a0e-4888-ac2f-bd8ebc43f327 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:11.023648 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2739add-0a0e-4888-ac2f-bd8ebc43f327 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:11.044891 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 783/1583] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:39:10 2022] POST /identity/v3/users => generated 284 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:39:11.046090 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaff86c-0dde-411b-a53d-c27f41db099c None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:11.046378 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaff86c-0dde-411b-a53d-c27f41db099c None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:11.046711 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaff86c-0dde-411b-a53d-c27f41db099c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:11.065333 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-5eaff86c-0dde-411b-a53d-c27f41db099c None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:11.066536 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-5eaff86c-0dde-411b-a53d-c27f41db099c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:11.074275 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 784/1584] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:39:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:39:11.078960 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaff86c-0dde-411b-a53d-c27f41db099c None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:11.085392 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaff86c-0dde-411b-a53d-c27f41db099c None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:11.090162 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172b2e4d-0386-434f-a248-604cbf4d6a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:11.090824 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172b2e4d-0386-434f-a248-604cbf4d6a05 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:11.091110 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172b2e4d-0386-434f-a248-604cbf4d6a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:11.091390 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172b2e4d-0386-434f-a248-604cbf4d6a05 admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:11.092109 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172b2e4d-0386-434f-a248-604cbf4d6a05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:11.094109 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172b2e4d-0386-434f-a248-604cbf4d6a05 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:11.101113 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 801/1585] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:39:11 2022] POST /identity/v3/users => generated 302 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:39:11.102402 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8868a6-c03e-4ee7-bbbe-5a0d89e87cfc None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:11.102695 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8868a6-c03e-4ee7-bbbe-5a0d89e87cfc None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:11.103921 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8868a6-c03e-4ee7-bbbe-5a0d89e87cfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:11.120262 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-de8868a6-c03e-4ee7-bbbe-5a0d89e87cfc None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:11.121469 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-de8868a6-c03e-4ee7-bbbe-5a0d89e87cfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:11.130262 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 802/1586] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:39:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:39:11.131008 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8868a6-c03e-4ee7-bbbe-5a0d89e87cfc None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:11.141572 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8868a6-c03e-4ee7-bbbe-5a0d89e87cfc None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:11.146389 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06656148-4ea5-482b-8846-a1a33787ca32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:11.148966 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06656148-4ea5-482b-8846-a1a33787ca32 admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:11.148966 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06656148-4ea5-482b-8846-a1a33787ca32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:11.148966 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06656148-4ea5-482b-8846-a1a33787ca32 admin admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:11.148966 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06656148-4ea5-482b-8846-a1a33787ca32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:11.150644 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06656148-4ea5-482b-8846-a1a33787ca32 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:11.193615 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 785/1587] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:39:11 2022] POST /identity/v3/users => generated 297 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:39:11.194447 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172b2e4d-0386-434f-a248-604cbf4d6a05 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:11.203755 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172b2e4d-0386-434f-a248-604cbf4d6a05 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:11.208228 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ec0efa-66ba-4372-8323-a6ca1d3ddcb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:11.208901 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ec0efa-66ba-4372-8323-a6ca1d3ddcb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:11.209200 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ec0efa-66ba-4372-8323-a6ca1d3ddcb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:11.209455 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ec0efa-66ba-4372-8323-a6ca1d3ddcb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:11.210164 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ec0efa-66ba-4372-8323-a6ca1d3ddcb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:11.211740 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ec0efa-66ba-4372-8323-a6ca1d3ddcb1 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:11.223168 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 786/1588] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:39:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:39:11.223992 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ec0efa-66ba-4372-8323-a6ca1d3ddcb1 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:11.233430 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ec0efa-66ba-4372-8323-a6ca1d3ddcb1 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:11.238640 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b629c85-8c6b-4bcc-9a66-64595ee99f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:11.241540 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b629c85-8c6b-4bcc-9a66-64595ee99f90 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:11.241540 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b629c85-8c6b-4bcc-9a66-64595ee99f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:11.241540 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b629c85-8c6b-4bcc-9a66-64595ee99f90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:11.241540 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b629c85-8c6b-4bcc-9a66-64595ee99f90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:11.242155 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b629c85-8c6b-4bcc-9a66-64595ee99f90 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:11.252894 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 787/1589] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:39:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:39:11.253719 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b629c85-8c6b-4bcc-9a66-64595ee99f90 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:11.264057 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b629c85-8c6b-4bcc-9a66-64595ee99f90 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:11.269054 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 803/1590] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:39:11 2022] POST /identity/v3/users => generated 283 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:39:11.269892 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06656148-4ea5-482b-8846-a1a33787ca32 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:11.270542 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87308568-69ce-4c28-9c0b-b6f5192f347f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:11.271534 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87308568-69ce-4c28-9c0b-b6f5192f347f admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:11.272028 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87308568-69ce-4c28-9c0b-b6f5192f347f admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:11.272604 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87308568-69ce-4c28-9c0b-b6f5192f347f admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:11.276087 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87308568-69ce-4c28-9c0b-b6f5192f347f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:11.276087 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87308568-69ce-4c28-9c0b-b6f5192f347f admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:11.283322 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=89,keystone:INSERT=11 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:39:11.285918 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 788/1591] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:39:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:39:11.286748 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87308568-69ce-4c28-9c0b-b6f5192f347f admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:11.291718 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:39:11.296877 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87308568-69ce-4c28-9c0b-b6f5192f347f admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:11.301536 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aa7e63-3eec-4458-a4e1-ddb1f33dde5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:11.304739 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa7e63-3eec-4458-a4e1-ddb1f33dde5d admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:11.304739 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa7e63-3eec-4458-a4e1-ddb1f33dde5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:11.304739 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa7e63-3eec-4458-a4e1-ddb1f33dde5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:11.304739 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aa7e63-3eec-4458-a4e1-ddb1f33dde5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:11.305133 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aa7e63-3eec-4458-a4e1-ddb1f33dde5d admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:11.314833 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 789/1592] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:39:11 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:39:11.315646 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aa7e63-3eec-4458-a4e1-ddb1f33dde5d admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:11.318736 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-18aa7e63-3eec-4458-a4e1-ddb1f33dde5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:11.332934 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06656148-4ea5-482b-8846-a1a33787ca32 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:11.339450 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab891b0e-0d40-42b4-8452-fffe881a632f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:11.339450 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab891b0e-0d40-42b4-8452-fffe881a632f service neutron] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:11.339450 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab891b0e-0d40-42b4-8452-fffe881a632f service neutron] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:11.339450 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab891b0e-0d40-42b4-8452-fffe881a632f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:11.348193 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab891b0e-0d40-42b4-8452-fffe881a632f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:11.349666 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab891b0e-0d40-42b4-8452-fffe881a632f service neutron] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:11.359258 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 804/1593] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:39:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:39:11.360072 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab891b0e-0d40-42b4-8452-fffe881a632f service neutron] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:11.369284 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab891b0e-0d40-42b4-8452-fffe881a632f service neutron] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:11.374355 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7a45e7-a9be-49e8-a0d1-3115de9a7198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:11.375060 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7a45e7-a9be-49e8-a0d1-3115de9a7198 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:11.375349 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7a45e7-a9be-49e8-a0d1-3115de9a7198 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:11.375627 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7a45e7-a9be-49e8-a0d1-3115de9a7198 admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:11.376363 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7a45e7-a9be-49e8-a0d1-3115de9a7198 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:11.377997 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7a45e7-a9be-49e8-a0d1-3115de9a7198 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:11.382098 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aa7e63-3eec-4458-a4e1-ddb1f33dde5d admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:11.386347 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6de159-ae59-4518-9600-9bce0a7de624 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:11.387268 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6de159-ae59-4518-9600-9bce0a7de624 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:11.387552 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6de159-ae59-4518-9600-9bce0a7de624 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:11.387828 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6de159-ae59-4518-9600-9bce0a7de624 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:11.389560 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 805/1594] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:39:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:39:11.391151 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7a45e7-a9be-49e8-a0d1-3115de9a7198 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:11.400179 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6de159-ae59-4518-9600-9bce0a7de624 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:11.401428 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7a45e7-a9be-49e8-a0d1-3115de9a7198 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:11.401615 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6de159-ae59-4518-9600-9bce0a7de624 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:11.406651 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419bdbd8-fde1-4710-a632-18447d4b9c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:11.407528 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419bdbd8-fde1-4710-a632-18447d4b9c81 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:11.407661 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419bdbd8-fde1-4710-a632-18447d4b9c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:11.407875 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419bdbd8-fde1-4710-a632-18447d4b9c81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:11.408706 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419bdbd8-fde1-4710-a632-18447d4b9c81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:11.410296 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419bdbd8-fde1-4710-a632-18447d4b9c81 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:11.412811 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 790/1595] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:39:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:39:11.413399 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6de159-ae59-4518-9600-9bce0a7de624 service neutron] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:11.423303 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6de159-ae59-4518-9600-9bce0a7de624 service neutron] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:11.429419 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 806/1596] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:39:11 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:39:11.429419 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419bdbd8-fde1-4710-a632-18447d4b9c81 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:11.431568 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e2337b-8de3-4a60-9b30-96047e6074c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:11.432528 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e2337b-8de3-4a60-9b30-96047e6074c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:11.432815 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e2337b-8de3-4a60-9b30-96047e6074c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:11.433101 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e2337b-8de3-4a60-9b30-96047e6074c4 admin admin] PATH_INFO: `/v3/projects/01ea22e1fb6c4f378f9ff4b59ae609dd/users/cee40d2493384405996c822271013d23/roles/3771341b3be641139e752aab690397a4` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:11.439989 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419bdbd8-fde1-4710-a632-18447d4b9c81 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:11.445110 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f786a7-cb67-48b5-b211-e07fb0519f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:11.445800 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f786a7-cb67-48b5-b211-e07fb0519f00 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:11.446186 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f786a7-cb67-48b5-b211-e07fb0519f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:11.446469 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f786a7-cb67-48b5-b211-e07fb0519f00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:11.447356 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f786a7-cb67-48b5-b211-e07fb0519f00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:11.449048 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f786a7-cb67-48b5-b211-e07fb0519f00 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:11.454282 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e2337b-8de3-4a60-9b30-96047e6074c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01ea22e1fb6c4f378f9ff4b59ae609dd, user_id=cee40d2493384405996c822271013d23, role_id=3771341b3be641139e752aab690397a4)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:11.454282 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: WARNING py.warnings [None req-c2e2337b-8de3-4a60-9b30-96047e6074c4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 14:39:11.454282 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: warnings.warn(msg) Jun 07 14:39:11.454282 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: Jun 07 14:39:11.454282 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e2337b-8de3-4a60-9b30-96047e6074c4 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:11.459365 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 807/1597] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:39:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:39:11.460184 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f786a7-cb67-48b5-b211-e07fb0519f00 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:11.470183 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f786a7-cb67-48b5-b211-e07fb0519f00 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:11.475453 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3984bfb8-1d1a-4515-8096-16deb56eb91d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:11.476158 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3984bfb8-1d1a-4515-8096-16deb56eb91d admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:11.476495 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3984bfb8-1d1a-4515-8096-16deb56eb91d admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:11.476767 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3984bfb8-1d1a-4515-8096-16deb56eb91d admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:11.477579 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3984bfb8-1d1a-4515-8096-16deb56eb91d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:11.479284 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3984bfb8-1d1a-4515-8096-16deb56eb91d admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:11.490854 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 808/1598] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:39:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:39:11.491818 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3984bfb8-1d1a-4515-8096-16deb56eb91d admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:11.514966 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3984bfb8-1d1a-4515-8096-16deb56eb91d admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:11.521249 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574b3819-2f1e-4498-b800-7e73675b9c6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:11.523328 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574b3819-2f1e-4498-b800-7e73675b9c6d admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:11.523328 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574b3819-2f1e-4498-b800-7e73675b9c6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:11.523328 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574b3819-2f1e-4498-b800-7e73675b9c6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:11.523328 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574b3819-2f1e-4498-b800-7e73675b9c6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:11.525396 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574b3819-2f1e-4498-b800-7e73675b9c6d admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:11.539772 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 809/1599] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:39:11 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:39:11.541611 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574b3819-2f1e-4498-b800-7e73675b9c6d admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:11.551589 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 791/1600] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:39:11 2022] PUT /identity/v3/projects/01ea22e1fb6c4f378f9ff4b59ae609dd/users/cee40d2493384405996c822271013d23/roles/3771341b3be641139e752aab690397a4 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:39:11.563818 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e2337b-8de3-4a60-9b30-96047e6074c4 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:11.569811 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574b3819-2f1e-4498-b800-7e73675b9c6d admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:11.647723 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7675a585-0b7f-47c3-9e7e-e82f26d4ca4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:11.648440 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7675a585-0b7f-47c3-9e7e-e82f26d4ca4c admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:11.648726 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7675a585-0b7f-47c3-9e7e-e82f26d4ca4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:11.649006 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7675a585-0b7f-47c3-9e7e-e82f26d4ca4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:11.649744 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7675a585-0b7f-47c3-9e7e-e82f26d4ca4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:11.651412 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7675a585-0b7f-47c3-9e7e-e82f26d4ca4c admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:11.652278 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e2337b-8de3-4a60-9b30-96047e6074c4 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:11.657652 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7530f74-5b10-43e5-8e7d-91e9be8e09cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:11.658550 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7530f74-5b10-43e5-8e7d-91e9be8e09cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:11.658837 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7530f74-5b10-43e5-8e7d-91e9be8e09cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:11.659191 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7530f74-5b10-43e5-8e7d-91e9be8e09cd admin admin] PATH_INFO: `/v3/projects/8eae64ee53144e4f8d583bb4288c0aeb/users/34eb9ec1f3ff44bbb03cf934cb2a04de/roles/3771341b3be641139e752aab690397a4` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:11.666636 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 810/1601] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:39:11 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:39:11.666636 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7675a585-0b7f-47c3-9e7e-e82f26d4ca4c admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:11.674521 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7675a585-0b7f-47c3-9e7e-e82f26d4ca4c admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:11.677264 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7530f74-5b10-43e5-8e7d-91e9be8e09cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=8eae64ee53144e4f8d583bb4288c0aeb, user_id=34eb9ec1f3ff44bbb03cf934cb2a04de, role_id=3771341b3be641139e752aab690397a4)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:11.679573 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fbf3a3-25d2-4376-a98f-bc1a78149bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:11.679919 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7530f74-5b10-43e5-8e7d-91e9be8e09cd admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:11.680234 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fbf3a3-25d2-4376-a98f-bc1a78149bc5 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:11.680568 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fbf3a3-25d2-4376-a98f-bc1a78149bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:11.680828 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fbf3a3-25d2-4376-a98f-bc1a78149bc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:11.681563 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fbf3a3-25d2-4376-a98f-bc1a78149bc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:11.683432 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fbf3a3-25d2-4376-a98f-bc1a78149bc5 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:11.693661 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 811/1602] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:39:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:39:11.697404 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fbf3a3-25d2-4376-a98f-bc1a78149bc5 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:11.705054 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fbf3a3-25d2-4376-a98f-bc1a78149bc5 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:11.709618 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f05b3f-561b-4af1-a334-098c193aee52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:11.710500 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f05b3f-561b-4af1-a334-098c193aee52 admin admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:11.710781 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f05b3f-561b-4af1-a334-098c193aee52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:11.711055 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f05b3f-561b-4af1-a334-098c193aee52 admin admin] PATH_INFO: `/v3/projects/d8b2399590184f02b899e7bc739bc466/users/b7f92cefd82440ed8368e2b388a4bc4c/roles/3771341b3be641139e752aab690397a4` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:11.735101 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f05b3f-561b-4af1-a334-098c193aee52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8b2399590184f02b899e7bc739bc466, user_id=b7f92cefd82440ed8368e2b388a4bc4c, role_id=3771341b3be641139e752aab690397a4)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:11.737919 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f05b3f-561b-4af1-a334-098c193aee52 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:11.820142 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 792/1603] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:39:11 2022] PUT /identity/v3/projects/8eae64ee53144e4f8d583bb4288c0aeb/users/34eb9ec1f3ff44bbb03cf934cb2a04de/roles/3771341b3be641139e752aab690397a4 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:39:11.821558 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7530f74-5b10-43e5-8e7d-91e9be8e09cd admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:11.882966 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 812/1604] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:39:11 2022] PUT /identity/v3/projects/d8b2399590184f02b899e7bc739bc466/users/b7f92cefd82440ed8368e2b388a4bc4c/roles/3771341b3be641139e752aab690397a4 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:39:11.884655 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0834c15-cb97-4958-8a55-c19e51fe46af None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:11.884952 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0834c15-cb97-4958-8a55-c19e51fe46af None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:11.885231 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0834c15-cb97-4958-8a55-c19e51fe46af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:11.900191 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:39:11.904221 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7530f74-5b10-43e5-8e7d-91e9be8e09cd admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:11.916358 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-a0834c15-cb97-4958-8a55-c19e51fe46af None None] MFA Rules not processed for user `cee40d2493384405996c822271013d23`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:11.917647 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-a0834c15-cb97-4958-8a55-c19e51fe46af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:11.923256 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:39:12.001636 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e44589-c109-48c2-a18c-3b47eab4ed52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:12.002685 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e44589-c109-48c2-a18c-3b47eab4ed52 admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:12.002972 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e44589-c109-48c2-a18c-3b47eab4ed52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:12.003266 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e44589-c109-48c2-a18c-3b47eab4ed52 admin admin] PATH_INFO: `/v3/projects/afb6ab205b714752bacfcd0e35f9f05e/users/a95e8ef758b440e7b4625c742edb4522/roles/3771341b3be641139e752aab690397a4` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:12.020609 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e44589-c109-48c2-a18c-3b47eab4ed52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afb6ab205b714752bacfcd0e35f9f05e, user_id=a95e8ef758b440e7b4625c742edb4522, role_id=3771341b3be641139e752aab690397a4)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:12.023710 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e44589-c109-48c2-a18c-3b47eab4ed52 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:12.025903 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 813/1605] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:39:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:39:12.026738 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0834c15-cb97-4958-8a55-c19e51fe46af None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:12.045176 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0834c15-cb97-4958-8a55-c19e51fe46af None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:12.051999 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a42ab28-98f5-4816-ab6c-108dfab840db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:12.056738 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a42ab28-98f5-4816-ab6c-108dfab840db admin admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:12.056738 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a42ab28-98f5-4816-ab6c-108dfab840db admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:12.056738 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a42ab28-98f5-4816-ab6c-108dfab840db admin admin] PATH_INFO: `/v3/projects/e80aa19d008640df9a1f924a7c03f7f6/users/48bbf88b32774d5da9ef93ac75686c34/roles/3771341b3be641139e752aab690397a4` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:12.084866 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a42ab28-98f5-4816-ab6c-108dfab840db admin admin] RBAC: Authorizing `identity:create_grant(project_id=e80aa19d008640df9a1f924a7c03f7f6, user_id=48bbf88b32774d5da9ef93ac75686c34, role_id=3771341b3be641139e752aab690397a4)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:12.088056 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a42ab28-98f5-4816-ab6c-108dfab840db admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:12.109142 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 793/1606] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:39:11 2022] PUT /identity/v3/projects/afb6ab205b714752bacfcd0e35f9f05e/users/a95e8ef758b440e7b4625c742edb4522/roles/3771341b3be641139e752aab690397a4 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:39:12.109142 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e44589-c109-48c2-a18c-3b47eab4ed52 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:12.160089 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 814/1607] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:39:12 2022] PUT /identity/v3/projects/e80aa19d008640df9a1f924a7c03f7f6/users/48bbf88b32774d5da9ef93ac75686c34/roles/3771341b3be641139e752aab690397a4 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:39:12.161567 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453bb015-2ea6-4e53-b428-f4f240f43fce None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:12.161959 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453bb015-2ea6-4e53-b428-f4f240f43fce None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:12.162502 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453bb015-2ea6-4e53-b428-f4f240f43fce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:12.173862 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e44589-c109-48c2-a18c-3b47eab4ed52 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:12.181036 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-453bb015-2ea6-4e53-b428-f4f240f43fce None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:12.182228 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-453bb015-2ea6-4e53-b428-f4f240f43fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:12.245880 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aadb76-5981-4f50-ad3f-b5aa83cb1f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:12.246777 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aadb76-5981-4f50-ad3f-b5aa83cb1f86 admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:12.247067 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aadb76-5981-4f50-ad3f-b5aa83cb1f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:12.247346 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aadb76-5981-4f50-ad3f-b5aa83cb1f86 admin admin] PATH_INFO: `/v3/projects/f26f06142ceb45088bb0c1b74aee4dc0/users/2b3e29113ef24adf95dc247bdabf7741/roles/3771341b3be641139e752aab690397a4` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:12.249414 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 815/1608] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:39:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:39:12.251410 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50265593-a711-45cc-8abc-c149aca2a17b None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:12.251708 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50265593-a711-45cc-8abc-c149aca2a17b None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:12.251990 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50265593-a711-45cc-8abc-c149aca2a17b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:12.273196 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46aadb76-5981-4f50-ad3f-b5aa83cb1f86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f26f06142ceb45088bb0c1b74aee4dc0, user_id=2b3e29113ef24adf95dc247bdabf7741, role_id=3771341b3be641139e752aab690397a4)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:12.275883 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46aadb76-5981-4f50-ad3f-b5aa83cb1f86 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:12.284534 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-50265593-a711-45cc-8abc-c149aca2a17b None None] MFA Rules not processed for user `b7f92cefd82440ed8368e2b388a4bc4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:12.285736 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-50265593-a711-45cc-8abc-c149aca2a17b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:12.336635 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 794/1609] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:39:12 2022] PUT /identity/v3/projects/f26f06142ceb45088bb0c1b74aee4dc0/users/2b3e29113ef24adf95dc247bdabf7741/roles/3771341b3be641139e752aab690397a4 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:39:12.338443 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ddb832-2e96-4bae-916d-c3dab06ffe83 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:12.338739 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ddb832-2e96-4bae-916d-c3dab06ffe83 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:12.339019 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ddb832-2e96-4bae-916d-c3dab06ffe83 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:12.340591 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 795/1610] 10.222.0.85 () {54 vars in 961 bytes} [Tue Jun 7 14:39:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:39:12.341650 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee713a2-0347-4b38-9cf8-8466c93dcf71 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:12.342240 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee713a2-0347-4b38-9cf8-8466c93dcf71 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:12.342240 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee713a2-0347-4b38-9cf8-8466c93dcf71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:12.369480 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:39:12.371337 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 816/1611] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:39:12 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:39:12.372889 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d86ff7-3b0c-4d8e-b80d-9f40306f0897 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:12.373386 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d86ff7-3b0c-4d8e-b80d-9f40306f0897 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:12.373626 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d86ff7-3b0c-4d8e-b80d-9f40306f0897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:12.375757 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-5ee713a2-0347-4b38-9cf8-8466c93dcf71 None None] MFA Rules not processed for user `a95e8ef758b440e7b4625c742edb4522`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:12.378680 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-5ee713a2-0347-4b38-9cf8-8466c93dcf71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:12.395080 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-54d86ff7-3b0c-4d8e-b80d-9f40306f0897 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:12.396339 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-54d86ff7-3b0c-4d8e-b80d-9f40306f0897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:12.404087 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:39:12.462350 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 796/1612] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:39:12 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:39:12.463664 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee713a2-0347-4b38-9cf8-8466c93dcf71 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:12.472792 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 817/1613] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:39:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:39:12.472792 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8289f3-15a1-432f-b3bc-63abc97dedcf None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:12.472792 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8289f3-15a1-432f-b3bc-63abc97dedcf None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:12.472792 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8289f3-15a1-432f-b3bc-63abc97dedcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:12.507505 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-1f8289f3-15a1-432f-b3bc-63abc97dedcf None None] MFA Rules not processed for user `2b3e29113ef24adf95dc247bdabf7741`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:12.509024 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-1f8289f3-15a1-432f-b3bc-63abc97dedcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:12.538553 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee713a2-0347-4b38-9cf8-8466c93dcf71 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:12.543920 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75827436-b1d8-4675-8ed1-633405928deb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:12.544751 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75827436-b1d8-4675-8ed1-633405928deb service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:12.545627 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75827436-b1d8-4675-8ed1-633405928deb service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:12.545896 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75827436-b1d8-4675-8ed1-633405928deb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:12.556713 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75827436-b1d8-4675-8ed1-633405928deb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:12.558556 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75827436-b1d8-4675-8ed1-633405928deb service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:12.570281 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 797/1614] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:39:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:39:12.571531 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a7f247-d4f7-4458-bf1f-4e256496bb67 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:12.571827 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a7f247-d4f7-4458-bf1f-4e256496bb67 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:12.572119 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a7f247-d4f7-4458-bf1f-4e256496bb67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:12.588538 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 818/1615] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:39:12 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:39:12.589906 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8289f3-15a1-432f-b3bc-63abc97dedcf None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:12.600308 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-d8a7f247-d4f7-4458-bf1f-4e256496bb67 None None] MFA Rules not processed for user `8e857db4902e42989d0d040bbc8517c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:12.601724 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-d8a7f247-d4f7-4458-bf1f-4e256496bb67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:12.649594 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8289f3-15a1-432f-b3bc-63abc97dedcf None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:12.653401 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07715833-aab9-4f1c-bf67-7186fc795b94 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d81f597cb44c56a966537089bcce4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:12.654059 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07715833-aab9-4f1c-bf67-7186fc795b94 service cinder] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:12.654349 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07715833-aab9-4f1c-bf67-7186fc795b94 service cinder] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:12.654633 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07715833-aab9-4f1c-bf67-7186fc795b94 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:12.673104 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 798/1616] 10.222.0.85 () {58 vars in 1058 bytes} [Tue Jun 7 14:39:12 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:39:12.673984 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a7f247-d4f7-4458-bf1f-4e256496bb67 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:12.685174 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a7f247-d4f7-4458-bf1f-4e256496bb67 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:12.690754 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f360e08e-60f1-4c83-b797-68d5bccaf72b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:12.691436 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f360e08e-60f1-4c83-b797-68d5bccaf72b admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:12.691622 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f360e08e-60f1-4c83-b797-68d5bccaf72b admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:12.691889 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f360e08e-60f1-4c83-b797-68d5bccaf72b admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:12.692628 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f360e08e-60f1-4c83-b797-68d5bccaf72b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:12.694643 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f360e08e-60f1-4c83-b797-68d5bccaf72b admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:12.722652 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:39:12.725546 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07715833-aab9-4f1c-bf67-7186fc795b94 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:12.727124 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07715833-aab9-4f1c-bf67-7186fc795b94 service cinder] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:12.738225 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 819/1617] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:39:12 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:39:12.739105 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07715833-aab9-4f1c-bf67-7186fc795b94 service cinder] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:12.749460 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07715833-aab9-4f1c-bf67-7186fc795b94 service cinder] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:12.754828 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5b273d-cf54-4a27-9319-5044bfadceaa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:12.756178 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5b273d-cf54-4a27-9319-5044bfadceaa service neutron] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:12.756794 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5b273d-cf54-4a27-9319-5044bfadceaa service neutron] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:12.757231 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5b273d-cf54-4a27-9319-5044bfadceaa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:12.768943 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5b273d-cf54-4a27-9319-5044bfadceaa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:12.775556 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5b273d-cf54-4a27-9319-5044bfadceaa service neutron] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:12.782137 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 820/1618] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:39:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:39:12.783049 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5b273d-cf54-4a27-9319-5044bfadceaa service neutron] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:12.794371 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5b273d-cf54-4a27-9319-5044bfadceaa service neutron] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:12.798805 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 799/1619] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:39:12 2022] POST /identity/v3/projects => generated 405 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:39:12.801434 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f360e08e-60f1-4c83-b797-68d5bccaf72b admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:12.804956 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7dadbf-419f-4d0f-adf1-36fd2e97c5e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:12.805757 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7dadbf-419f-4d0f-adf1-36fd2e97c5e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:12.806204 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7dadbf-419f-4d0f-adf1-36fd2e97c5e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:12.806494 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7dadbf-419f-4d0f-adf1-36fd2e97c5e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:12.808125 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7dadbf-419f-4d0f-adf1-36fd2e97c5e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:12.810243 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7dadbf-419f-4d0f-adf1-36fd2e97c5e5 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:12.852395 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:39:12.886166 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 821/1620] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:39:12 2022] POST /identity/v3/projects => generated 363 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:39:12.887440 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7dadbf-419f-4d0f-adf1-36fd2e97c5e5 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:12.925421 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f360e08e-60f1-4c83-b797-68d5bccaf72b admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:13.020207 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7dadbf-419f-4d0f-adf1-36fd2e97c5e5 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:13.028229 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dde5c4-a1d7-44a4-aacc-a873aca22ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:13.029112 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dde5c4-a1d7-44a4-aacc-a873aca22ead admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:13.029460 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dde5c4-a1d7-44a4-aacc-a873aca22ead admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:13.029857 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dde5c4-a1d7-44a4-aacc-a873aca22ead admin admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:13.030738 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dde5c4-a1d7-44a4-aacc-a873aca22ead admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:13.033505 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480f5058-9f3e-4837-9efb-c1a7c28c61f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e857db4902e42989d0d040bbc8517c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:13.034241 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dde5c4-a1d7-44a4-aacc-a873aca22ead admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:13.034558 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480f5058-9f3e-4837-9efb-c1a7c28c61f5 service nova] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:13.034998 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480f5058-9f3e-4837-9efb-c1a7c28c61f5 service nova] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:13.036631 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480f5058-9f3e-4837-9efb-c1a7c28c61f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:13.123161 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480f5058-9f3e-4837-9efb-c1a7c28c61f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:13.126687 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480f5058-9f3e-4837-9efb-c1a7c28c61f5 service nova] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:13.141632 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 800/1621] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:39:12 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:39:13.142562 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480f5058-9f3e-4837-9efb-c1a7c28c61f5 service nova] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:13.142987 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 822/1622] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:39:12 2022] POST /identity/v3/users => generated 309 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:39:13.146918 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dde5c4-a1d7-44a4-aacc-a873aca22ead admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:13.153835 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480f5058-9f3e-4837-9efb-c1a7c28c61f5 service nova] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:13.161869 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71066278-f506-4ade-87c5-bdbb963a9c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:13.163437 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71066278-f506-4ade-87c5-bdbb963a9c39 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:13.163437 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71066278-f506-4ade-87c5-bdbb963a9c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:13.163437 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71066278-f506-4ade-87c5-bdbb963a9c39 admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:13.164329 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71066278-f506-4ade-87c5-bdbb963a9c39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:13.166010 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dde5c4-a1d7-44a4-aacc-a873aca22ead admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:13.167727 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71066278-f506-4ade-87c5-bdbb963a9c39 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:13.172628 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8ba4f0-235e-42e1-bef2-11d1000ac3f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:13.173583 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8ba4f0-235e-42e1-bef2-11d1000ac3f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:13.174113 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8ba4f0-235e-42e1-bef2-11d1000ac3f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:13.174713 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8ba4f0-235e-42e1-bef2-11d1000ac3f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:13.175677 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8ba4f0-235e-42e1-bef2-11d1000ac3f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:13.177534 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8ba4f0-235e-42e1-bef2-11d1000ac3f9 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:13.191377 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 823/1623] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:39:13 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:39:13.195487 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8ba4f0-235e-42e1-bef2-11d1000ac3f9 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:13.211157 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8ba4f0-235e-42e1-bef2-11d1000ac3f9 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:13.224577 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f33c77-be5f-4855-bf94-5382923f1055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:13.224577 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f33c77-be5f-4855-bf94-5382923f1055 admin admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:13.224577 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f33c77-be5f-4855-bf94-5382923f1055 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:13.224577 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f33c77-be5f-4855-bf94-5382923f1055 admin admin] PATH_INFO: `/v3/projects/d547ba75dd9542368772362f9f13e72d/users/85330014a69f449db1714b216c41f852/roles/9baff2b18f2649f7b757ff2e1f63c853` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:13.278821 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f33c77-be5f-4855-bf94-5382923f1055 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d547ba75dd9542368772362f9f13e72d, user_id=85330014a69f449db1714b216c41f852, role_id=9baff2b18f2649f7b757ff2e1f63c853)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:13.286920 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f33c77-be5f-4855-bf94-5382923f1055 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:13.341509 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 801/1624] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:39:13 2022] POST /identity/v3/users => generated 288 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:39:13.343710 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f308bc-3d13-4664-bf08-31ff4f86bf7a None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:13.346487 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f308bc-3d13-4664-bf08-31ff4f86bf7a None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:13.347191 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f308bc-3d13-4664-bf08-31ff4f86bf7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:13.376794 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 824/1625] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:39:13 2022] PUT /identity/v3/projects/d547ba75dd9542368772362f9f13e72d/users/85330014a69f449db1714b216c41f852/roles/9baff2b18f2649f7b757ff2e1f63c853 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:39:13.383378 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f33c77-be5f-4855-bf94-5382923f1055 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:13.414515 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-d9f308bc-3d13-4664-bf08-31ff4f86bf7a None None] MFA Rules not processed for user `d800b8efe792477bac3ca063920a4a9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:13.421407 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-d9f308bc-3d13-4664-bf08-31ff4f86bf7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:13.503616 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:39:13.550398 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f33c77-be5f-4855-bf94-5382923f1055 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:13.558969 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dfa6cf-fc0f-4210-8e80-7da8cd4dac7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:13.560885 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dfa6cf-fc0f-4210-8e80-7da8cd4dac7e admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:13.563242 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dfa6cf-fc0f-4210-8e80-7da8cd4dac7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:13.563564 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dfa6cf-fc0f-4210-8e80-7da8cd4dac7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:13.564334 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dfa6cf-fc0f-4210-8e80-7da8cd4dac7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:13.573184 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dfa6cf-fc0f-4210-8e80-7da8cd4dac7e admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:13.590461 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 825/1626] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:39:13 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:39:13.591362 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dfa6cf-fc0f-4210-8e80-7da8cd4dac7e admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:13.603681 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 802/1627] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:39:13 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:39:13.603681 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d3fc1a-a128-41c5-a85b-d63faafcb916 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:13.603681 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d3fc1a-a128-41c5-a85b-d63faafcb916 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:13.603681 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d3fc1a-a128-41c5-a85b-d63faafcb916 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:13.603681 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 803/1628] 10.222.0.85 () {54 vars in 912 bytes} [Tue Jun 7 14:39:13 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:39:13.603681 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d3fc1a-a128-41c5-a85b-d63faafcb916 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:13.617700 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dfa6cf-fc0f-4210-8e80-7da8cd4dac7e admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:13.627040 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf4c71b-c119-4160-af79-4e2fc1040e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:13.628283 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf4c71b-c119-4160-af79-4e2fc1040e5c admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:13.628882 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf4c71b-c119-4160-af79-4e2fc1040e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:13.629827 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf4c71b-c119-4160-af79-4e2fc1040e5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:13.631457 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf4c71b-c119-4160-af79-4e2fc1040e5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:13.634669 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf4c71b-c119-4160-af79-4e2fc1040e5c admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:13.638593 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d3fc1a-a128-41c5-a85b-d63faafcb916 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:13.643751 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:39:13.655635 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190c9589-fbea-4c3a-8d5f-ca223c1cbb17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:13.657259 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190c9589-fbea-4c3a-8d5f-ca223c1cbb17 admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:13.657742 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190c9589-fbea-4c3a-8d5f-ca223c1cbb17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:13.658417 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190c9589-fbea-4c3a-8d5f-ca223c1cbb17 admin admin] PATH_INFO: `/v3/projects/a03392d5ef2a4962b4513dbcbf93693f/users/3b50ca1c44764618ac031785d365038a/roles/9baff2b18f2649f7b757ff2e1f63c853` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:13.658729 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 826/1629] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:39:13 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:39:13.665099 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680d84cd-8418-4893-adf3-c58a4700edbe None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:13.665099 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680d84cd-8418-4893-adf3-c58a4700edbe None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:13.665099 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680d84cd-8418-4893-adf3-c58a4700edbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:13.695749 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190c9589-fbea-4c3a-8d5f-ca223c1cbb17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a03392d5ef2a4962b4513dbcbf93693f, user_id=3b50ca1c44764618ac031785d365038a, role_id=9baff2b18f2649f7b757ff2e1f63c853)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:13.699031 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190c9589-fbea-4c3a-8d5f-ca223c1cbb17 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:13.699673 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-680d84cd-8418-4893-adf3-c58a4700edbe None None] MFA Rules not processed for user `d800b8efe792477bac3ca063920a4a9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:13.703761 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-680d84cd-8418-4893-adf3-c58a4700edbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:13.718963 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 827/1630] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:39:13 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:39:13.718963 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9cac01-60fa-43fc-a1a2-65cc520ee78a None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:13.718963 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9cac01-60fa-43fc-a1a2-65cc520ee78a None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:13.718963 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9cac01-60fa-43fc-a1a2-65cc520ee78a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:13.742959 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-ea9cac01-60fa-43fc-a1a2-65cc520ee78a None None] MFA Rules not processed for user `8e857db4902e42989d0d040bbc8517c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:13.744592 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-ea9cac01-60fa-43fc-a1a2-65cc520ee78a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:13.791028 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 804/1631] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:39:13 2022] PUT /identity/v3/projects/a03392d5ef2a4962b4513dbcbf93693f/users/3b50ca1c44764618ac031785d365038a/roles/9baff2b18f2649f7b757ff2e1f63c853 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:39:13.791028 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190c9589-fbea-4c3a-8d5f-ca223c1cbb17 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:13.887203 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 828/1632] 10.222.0.85 () {58 vars in 1009 bytes} [Tue Jun 7 14:39:13 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:39:13.889198 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e016851c-58d6-42ce-b69b-89a8263a9009 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:13.890991 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e016851c-58d6-42ce-b69b-89a8263a9009 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:13.890991 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e016851c-58d6-42ce-b69b-89a8263a9009 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:13.893483 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 829/1633] 10.222.0.85 () {54 vars in 961 bytes} [Tue Jun 7 14:39:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:39:13.897794 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e016851c-58d6-42ce-b69b-89a8263a9009 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:13.935043 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190c9589-fbea-4c3a-8d5f-ca223c1cbb17 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:13.946820 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d62b704-42bc-4844-b657-4579993c1bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:13.947751 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d62b704-42bc-4844-b657-4579993c1bc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:13.948038 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d62b704-42bc-4844-b657-4579993c1bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:13.948321 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d62b704-42bc-4844-b657-4579993c1bc1 admin admin] PATH_INFO: `/v3/projects/d547ba75dd9542368772362f9f13e72d/users/85330014a69f449db1714b216c41f852/roles/9baff2b18f2649f7b757ff2e1f63c853` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:13.954597 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d62b704-42bc-4844-b657-4579993c1bc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d547ba75dd9542368772362f9f13e72d, user_id=85330014a69f449db1714b216c41f852, role_id=9baff2b18f2649f7b757ff2e1f63c853)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:13.957284 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d62b704-42bc-4844-b657-4579993c1bc1 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:14.013067 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 805/1634] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:39:13 2022] PUT /identity/v3/projects/d547ba75dd9542368772362f9f13e72d/users/85330014a69f449db1714b216c41f852/roles/9baff2b18f2649f7b757ff2e1f63c853 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:39:14.023665 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d62b704-42bc-4844-b657-4579993c1bc1 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:14.059973 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e016851c-58d6-42ce-b69b-89a8263a9009 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:14.174313 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630b4c7e-f8f4-4d83-b01f-367d521e484b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:14.175167 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b4c7e-f8f4-4d83-b01f-367d521e484b admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:14.175963 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b4c7e-f8f4-4d83-b01f-367d521e484b admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:14.177179 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b4c7e-f8f4-4d83-b01f-367d521e484b admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:14.178014 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630b4c7e-f8f4-4d83-b01f-367d521e484b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:14.182227 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630b4c7e-f8f4-4d83-b01f-367d521e484b admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:14.198514 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d62b704-42bc-4844-b657-4579993c1bc1 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:14.205341 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 830/1635] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:39:13 2022] GET /identity/v3/roles => generated 1420 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:39:14.206917 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b9a032-9cc5-43de-b56f-f84e8c7896ed None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:14.207223 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b9a032-9cc5-43de-b56f-f84e8c7896ed None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:14.207512 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b9a032-9cc5-43de-b56f-f84e8c7896ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:14.208038 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82742f06-4a65-48f7-8285-702b69288fdb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d81f597cb44c56a966537089bcce4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:14.211078 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82742f06-4a65-48f7-8285-702b69288fdb service cinder] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:14.211535 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82742f06-4a65-48f7-8285-702b69288fdb service cinder] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:14.212023 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82742f06-4a65-48f7-8285-702b69288fdb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:14.254748 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-93b9a032-9cc5-43de-b56f-f84e8c7896ed None None] MFA Rules not processed for user `8e857db4902e42989d0d040bbc8517c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:14.256178 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-93b9a032-9cc5-43de-b56f-f84e8c7896ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:14.292937 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:39:14.330931 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82742f06-4a65-48f7-8285-702b69288fdb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:14.333168 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82742f06-4a65-48f7-8285-702b69288fdb service cinder] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:14.353587 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 806/1636] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:39:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:39:14.356859 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74895a3-0781-402d-8ced-bdf40a1226c1 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:14.357162 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74895a3-0781-402d-8ced-bdf40a1226c1 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:14.357780 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74895a3-0781-402d-8ced-bdf40a1226c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:14.366437 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:39:14.395376 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 831/1637] 10.222.0.85 () {58 vars in 1058 bytes} [Tue Jun 7 14:39:14 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:39:14.399345 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-d74895a3-0781-402d-8ced-bdf40a1226c1 None None] MFA Rules not processed for user `85330014a69f449db1714b216c41f852`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:14.400818 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-d74895a3-0781-402d-8ced-bdf40a1226c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:14.410474 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b9a032-9cc5-43de-b56f-f84e8c7896ed None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:14.428160 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b9a032-9cc5-43de-b56f-f84e8c7896ed None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:14.442734 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c98bde-8ddd-4ca0-8769-bc32db671b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:14.443660 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c98bde-8ddd-4ca0-8769-bc32db671b97 admin admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:14.443948 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c98bde-8ddd-4ca0-8769-bc32db671b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:14.444231 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c98bde-8ddd-4ca0-8769-bc32db671b97 admin admin] PATH_INFO: `/v3/projects/a03392d5ef2a4962b4513dbcbf93693f/users/3b50ca1c44764618ac031785d365038a/roles/9baff2b18f2649f7b757ff2e1f63c853` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:14.449406 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c98bde-8ddd-4ca0-8769-bc32db671b97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a03392d5ef2a4962b4513dbcbf93693f, user_id=3b50ca1c44764618ac031785d365038a, role_id=9baff2b18f2649f7b757ff2e1f63c853)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:14.455125 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c98bde-8ddd-4ca0-8769-bc32db671b97 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:14.494592 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 832/1638] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:39:14 2022] PUT /identity/v3/projects/a03392d5ef2a4962b4513dbcbf93693f/users/3b50ca1c44764618ac031785d365038a/roles/9baff2b18f2649f7b757ff2e1f63c853 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:39:14.497687 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c98bde-8ddd-4ca0-8769-bc32db671b97 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:14.611917 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 807/1639] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:39:14 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:39:14.611917 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9003335-a97b-4096-9c43-2e533d49cab0 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:14.613107 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9003335-a97b-4096-9c43-2e533d49cab0 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:14.613419 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9003335-a97b-4096-9c43-2e533d49cab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:14.680926 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-b9003335-a97b-4096-9c43-2e533d49cab0 None None] MFA Rules not processed for user `3b50ca1c44764618ac031785d365038a`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:14.684335 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-b9003335-a97b-4096-9c43-2e533d49cab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:14.686968 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c98bde-8ddd-4ca0-8769-bc32db671b97 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:14.720053 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd48f424-9b49-4ad5-9f3e-4498e2c18ea7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e857db4902e42989d0d040bbc8517c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:14.720053 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd48f424-9b49-4ad5-9f3e-4498e2c18ea7 service nova] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:14.720053 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd48f424-9b49-4ad5-9f3e-4498e2c18ea7 service nova] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:14.720053 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd48f424-9b49-4ad5-9f3e-4498e2c18ea7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:14.816405 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd48f424-9b49-4ad5-9f3e-4498e2c18ea7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:14.818059 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd48f424-9b49-4ad5-9f3e-4498e2c18ea7 service nova] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:14.833358 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 833/1640] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:39:14 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:39:14.834246 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd48f424-9b49-4ad5-9f3e-4498e2c18ea7 service nova] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:14.836909 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 808/1641] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:39:14 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:39:14.899678 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:39:14.913848 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd48f424-9b49-4ad5-9f3e-4498e2c18ea7 service nova] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:14.920317 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d85e159-3479-403c-8e3f-b5e50340fabf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d81f597cb44c56a966537089bcce4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:14.921401 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d85e159-3479-403c-8e3f-b5e50340fabf service cinder] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:14.922359 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d85e159-3479-403c-8e3f-b5e50340fabf service cinder] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:14.922798 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d85e159-3479-403c-8e3f-b5e50340fabf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:15.050058 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d85e159-3479-403c-8e3f-b5e50340fabf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:15.051681 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d85e159-3479-403c-8e3f-b5e50340fabf service cinder] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:15.063758 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 834/1642] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:39:14 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:39:15.145913 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9003335-a97b-4096-9c43-2e533d49cab0 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:15.158679 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9003335-a97b-4096-9c43-2e533d49cab0 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:15.165347 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe385b8e-865d-4148-8011-2b3e41defec7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d81f597cb44c56a966537089bcce4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:15.168969 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe385b8e-865d-4148-8011-2b3e41defec7 service cinder] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:15.169210 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe385b8e-865d-4148-8011-2b3e41defec7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:15.170062 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe385b8e-865d-4148-8011-2b3e41defec7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:15.190739 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe385b8e-865d-4148-8011-2b3e41defec7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:15.192452 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe385b8e-865d-4148-8011-2b3e41defec7 service cinder] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:15.206173 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff23acc3-4939-4451-bc35-2cf303851b3c None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:15.207104 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff23acc3-4939-4451-bc35-2cf303851b3c None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:15.207592 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff23acc3-4939-4451-bc35-2cf303851b3c None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:15.209198 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 809/1643] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:39:15 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:39:15.211686 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 835/1644] 10.222.0.85 () {54 vars in 912 bytes} [Tue Jun 7 14:39:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:39:15.226517 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699fc822-78d6-4525-9abe-7d6c786af941 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:15.228996 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699fc822-78d6-4525-9abe-7d6c786af941 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:15.229834 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699fc822-78d6-4525-9abe-7d6c786af941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:15.254511 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-699fc822-78d6-4525-9abe-7d6c786af941 None None] MFA Rules not processed for user `8e857db4902e42989d0d040bbc8517c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:15.258502 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-699fc822-78d6-4525-9abe-7d6c786af941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:15.278895 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 810/1645] 10.222.0.85 () {58 vars in 1009 bytes} [Tue Jun 7 14:39:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:39:15.305360 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff23acc3-4939-4451-bc35-2cf303851b3c None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:15.307320 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-ff23acc3-4939-4451-bc35-2cf303851b3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:15.700283 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff23acc3-4939-4451-bc35-2cf303851b3c None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:15.700283 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70df8ea-fb32-4cf2-9dab-62fdbedecd66 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51da9867d9147f198b8077e953c4f9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:15.700283 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70df8ea-fb32-4cf2-9dab-62fdbedecd66 service glance] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:15.700283 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70df8ea-fb32-4cf2-9dab-62fdbedecd66 service glance] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:15.708814 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70df8ea-fb32-4cf2-9dab-62fdbedecd66 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:15.708814 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70df8ea-fb32-4cf2-9dab-62fdbedecd66 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:15.708814 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70df8ea-fb32-4cf2-9dab-62fdbedecd66 service glance] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:15.708814 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 836/1646] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:39:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:39:15.763236 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699fc822-78d6-4525-9abe-7d6c786af941 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:15.833844 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:39:15.899725 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699fc822-78d6-4525-9abe-7d6c786af941 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:15.913126 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f7a826-628a-4526-bac7-199e7d517bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:15.914361 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f7a826-628a-4526-bac7-199e7d517bf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:15.914775 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f7a826-628a-4526-bac7-199e7d517bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:15.916642 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f7a826-628a-4526-bac7-199e7d517bf4 admin admin] PATH_INFO: `/v3/users/a95e8ef758b440e7b4625c742edb4522` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:15.918741 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f7a826-628a-4526-bac7-199e7d517bf4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a95e8ef758b440e7b4625c742edb4522)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:15.920041 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: WARNING py.warnings [None req-22f7a826-628a-4526-bac7-199e7d517bf4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 14:39:15.920041 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: warnings.warn(msg) Jun 07 14:39:15.920041 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: Jun 07 14:39:15.922281 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f7a826-628a-4526-bac7-199e7d517bf4 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:16.166390 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-22f7a826-628a-4526-bac7-199e7d517bf4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a95e8ef758b440e7b4625c742edb4522'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:16.167746 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-22f7a826-628a-4526-bac7-199e7d517bf4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a95e8ef758b440e7b4625c742edb4522'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:16.168640 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-22f7a826-628a-4526-bac7-199e7d517bf4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a95e8ef758b440e7b4625c742edb4522'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:16.183149 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-22f7a826-628a-4526-bac7-199e7d517bf4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a95e8ef758b440e7b4625c742edb4522'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:16.241059 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-22f7a826-628a-4526-bac7-199e7d517bf4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a95e8ef758b440e7b4625c742edb4522'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:16.296246 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b06e2d-fcf2-4cf6-90f7-6e0dfc25ced0 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:16.296866 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b06e2d-fcf2-4cf6-90f7-6e0dfc25ced0 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:16.296866 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b06e2d-fcf2-4cf6-90f7-6e0dfc25ced0 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:16.298560 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 837/1647] 10.222.0.85 () {54 vars in 912 bytes} [Tue Jun 7 14:39:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:39:16.321606 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab25810-d261-4be1-9d05-7d437de0b988 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:16.321606 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab25810-d261-4be1-9d05-7d437de0b988 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:16.321606 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab25810-d261-4be1-9d05-7d437de0b988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:16.327362 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 811/1648] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:39:15 2022] DELETE /identity/v3/users/a95e8ef758b440e7b4625c742edb4522 => generated 0 bytes in 565 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:39:16.332466 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60186e91-eefe-4407-8475-1b4817ae2eaa None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:16.332663 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60186e91-eefe-4407-8475-1b4817ae2eaa None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:16.332941 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60186e91-eefe-4407-8475-1b4817ae2eaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:16.377658 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-60186e91-eefe-4407-8475-1b4817ae2eaa None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:16.379140 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-60186e91-eefe-4407-8475-1b4817ae2eaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:16.402769 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-0ab25810-d261-4be1-9d05-7d437de0b988 None None] MFA Rules not processed for user `7ac4c85918c045b793a1a53300c989a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:16.404252 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-0ab25810-d261-4be1-9d05-7d437de0b988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:16.536011 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 838/1649] 10.222.0.85 () {58 vars in 1009 bytes} [Tue Jun 7 14:39:16 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:39:16.567308 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 812/1650] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:39:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:39:16.602357 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab25810-d261-4be1-9d05-7d437de0b988 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:16.603807 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-0ab25810-d261-4be1-9d05-7d437de0b988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:16.620152 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab25810-d261-4be1-9d05-7d437de0b988 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:16.645913 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7310e770-309c-4056-9c44-23f462b70547 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:16.646665 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7310e770-309c-4056-9c44-23f462b70547 service neutron] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:16.646903 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7310e770-309c-4056-9c44-23f462b70547 service neutron] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:16.647181 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7310e770-309c-4056-9c44-23f462b70547 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:16.670655 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7310e770-309c-4056-9c44-23f462b70547 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:16.672105 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7310e770-309c-4056-9c44-23f462b70547 service neutron] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:16.693727 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60186e91-eefe-4407-8475-1b4817ae2eaa None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:16.695879 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-60186e91-eefe-4407-8475-1b4817ae2eaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:16.699010 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 839/1651] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:39:16 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:39:16.718588 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60186e91-eefe-4407-8475-1b4817ae2eaa None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:16.729072 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed51e9e-04e4-4ae2-8bed-a8af3cc5ce64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:16.730050 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed51e9e-04e4-4ae2-8bed-a8af3cc5ce64 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:16.730512 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed51e9e-04e4-4ae2-8bed-a8af3cc5ce64 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:16.730952 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed51e9e-04e4-4ae2-8bed-a8af3cc5ce64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:16.756316 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed51e9e-04e4-4ae2-8bed-a8af3cc5ce64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:16.758214 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed51e9e-04e4-4ae2-8bed-a8af3cc5ce64 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:16.788592 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 813/1652] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:39:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 14:39:16.845961 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7310e770-309c-4056-9c44-23f462b70547 service neutron] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:16.847432 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-7310e770-309c-4056-9c44-23f462b70547 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:16.880308 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7310e770-309c-4056-9c44-23f462b70547 service neutron] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:16.885657 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313bda32-2a13-433e-b686-3a652e5d9cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:16.886552 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313bda32-2a13-433e-b686-3a652e5d9cff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:16.887270 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313bda32-2a13-433e-b686-3a652e5d9cff admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:16.887552 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313bda32-2a13-433e-b686-3a652e5d9cff admin admin] PATH_INFO: `/v3/users/2b3e29113ef24adf95dc247bdabf7741` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:16.888889 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313bda32-2a13-433e-b686-3a652e5d9cff admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b3e29113ef24adf95dc247bdabf7741)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:16.891431 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313bda32-2a13-433e-b686-3a652e5d9cff admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:16.931656 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:39:17.188241 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-313bda32-2a13-433e-b686-3a652e5d9cff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b3e29113ef24adf95dc247bdabf7741'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:17.189664 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-313bda32-2a13-433e-b686-3a652e5d9cff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b3e29113ef24adf95dc247bdabf7741'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:17.190710 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-313bda32-2a13-433e-b686-3a652e5d9cff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b3e29113ef24adf95dc247bdabf7741'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:17.208099 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-313bda32-2a13-433e-b686-3a652e5d9cff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b3e29113ef24adf95dc247bdabf7741'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:17.235447 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-313bda32-2a13-433e-b686-3a652e5d9cff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b3e29113ef24adf95dc247bdabf7741'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:17.363691 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 840/1653] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:39:16 2022] DELETE /identity/v3/users/2b3e29113ef24adf95dc247bdabf7741 => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:39:17.368289 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf08195-1156-49a3-b4bf-f81582124e93 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:17.368828 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf08195-1156-49a3-b4bf-f81582124e93 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:17.369101 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf08195-1156-49a3-b4bf-f81582124e93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:17.402124 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-caf08195-1156-49a3-b4bf-f81582124e93 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:17.403434 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-caf08195-1156-49a3-b4bf-f81582124e93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:17.451347 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:39:17.531743 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 814/1654] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:39:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:39:17.583837 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313bda32-2a13-433e-b686-3a652e5d9cff admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:17.585661 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-313bda32-2a13-433e-b686-3a652e5d9cff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:17.780047 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313bda32-2a13-433e-b686-3a652e5d9cff admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:17.801763 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c1741f-2f8c-4ff8-b56f-7ea4c53ab014 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:17.811351 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c1741f-2f8c-4ff8-b56f-7ea4c53ab014 service neutron] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:17.811718 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c1741f-2f8c-4ff8-b56f-7ea4c53ab014 service neutron] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:17.816868 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c1741f-2f8c-4ff8-b56f-7ea4c53ab014 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:17.853264 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c1741f-2f8c-4ff8-b56f-7ea4c53ab014 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:17.854723 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c1741f-2f8c-4ff8-b56f-7ea4c53ab014 service neutron] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:17.896995 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c132a259-fdcf-4c5e-86ab-9cf60b96a22a None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:17.897535 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c132a259-fdcf-4c5e-86ab-9cf60b96a22a None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:17.898068 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c132a259-fdcf-4c5e-86ab-9cf60b96a22a None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:17.900298 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 815/1655] 10.222.0.85 () {54 vars in 960 bytes} [Tue Jun 7 14:39:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 14:39:17.907691 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aab1a8-35cd-4cce-b4a8-b9f8a4a9d914 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:17.908167 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 841/1656] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:39:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:39:17.910688 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aab1a8-35cd-4cce-b4a8-b9f8a4a9d914 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:17.911176 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aab1a8-35cd-4cce-b4a8-b9f8a4a9d914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:17.994258 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-f8aab1a8-35cd-4cce-b4a8-b9f8a4a9d914 None None] MFA Rules not processed for user `6330294b1c8e4dcd85aad189f007b0aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:17.995998 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-f8aab1a8-35cd-4cce-b4a8-b9f8a4a9d914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:18.150327 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 816/1657] 10.222.0.85 () {58 vars in 1057 bytes} [Tue Jun 7 14:39:17 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:39:18.176997 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c1741f-2f8c-4ff8-b56f-7ea4c53ab014 service neutron] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:18.178449 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aab1a8-35cd-4cce-b4a8-b9f8a4a9d914 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:18.181558 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-f8aab1a8-35cd-4cce-b4a8-b9f8a4a9d914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:18.203786 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c1741f-2f8c-4ff8-b56f-7ea4c53ab014 service neutron] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:18.225007 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aab1a8-35cd-4cce-b4a8-b9f8a4a9d914 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:18.225641 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167d7bea-523a-4df4-923e-902ec51d036a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6330294b1c8e4dcd85aad189f007b0aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:18.226402 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167d7bea-523a-4df4-923e-902ec51d036a service placement] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:18.228318 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167d7bea-523a-4df4-923e-902ec51d036a service placement] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:18.230544 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167d7bea-523a-4df4-923e-902ec51d036a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:18.238271 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-167d7bea-523a-4df4-923e-902ec51d036a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:18.243262 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c7e469-539a-428e-a26e-519a2c2f44d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:18.244993 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c7e469-539a-428e-a26e-519a2c2f44d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:18.245439 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c7e469-539a-428e-a26e-519a2c2f44d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:18.245895 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c7e469-539a-428e-a26e-519a2c2f44d6 admin admin] PATH_INFO: `/v3/projects/afb6ab205b714752bacfcd0e35f9f05e` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:18.255474 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c7e469-539a-428e-a26e-519a2c2f44d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=afb6ab205b714752bacfcd0e35f9f05e)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:18.257799 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c7e469-539a-428e-a26e-519a2c2f44d6 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:18.283419 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167d7bea-523a-4df4-923e-902ec51d036a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:18.285847 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167d7bea-523a-4df4-923e-902ec51d036a service placement] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:18.339584 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 842/1658] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:39:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:39:18.508876 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-88c7e469-539a-428e-a26e-519a2c2f44d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afb6ab205b714752bacfcd0e35f9f05e', 'request_id': 'req-88c7e469-539a-428e-a26e-519a2c2f44d6'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:18.604255 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-88c7e469-539a-428e-a26e-519a2c2f44d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afb6ab205b714752bacfcd0e35f9f05e', 'request_id': 'req-88c7e469-539a-428e-a26e-519a2c2f44d6'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:18.614616 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-88c7e469-539a-428e-a26e-519a2c2f44d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afb6ab205b714752bacfcd0e35f9f05e', 'request_id': 'req-88c7e469-539a-428e-a26e-519a2c2f44d6'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:18.653559 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-88c7e469-539a-428e-a26e-519a2c2f44d6 admin admin] The token cache is being invalidate because project afb6ab205b714752bacfcd0e35f9f05e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:39:18.653883 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-88c7e469-539a-428e-a26e-519a2c2f44d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:18.655679 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-88c7e469-539a-428e-a26e-519a2c2f44d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:18.671433 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 817/1659] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:39:18 2022] DELETE /identity/v3/projects/afb6ab205b714752bacfcd0e35f9f05e => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:39:18.672003 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167d7bea-523a-4df4-923e-902ec51d036a service placement] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:18.688969 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167d7bea-523a-4df4-923e-902ec51d036a service placement] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:18.939808 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb365462-3dc1-49f5-a1d7-278e794dea39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:18.940838 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb365462-3dc1-49f5-a1d7-278e794dea39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:18.941149 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb365462-3dc1-49f5-a1d7-278e794dea39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:18.941401 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb365462-3dc1-49f5-a1d7-278e794dea39 admin admin] PATH_INFO: `/v3/users/85330014a69f449db1714b216c41f852` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:18.945580 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb365462-3dc1-49f5-a1d7-278e794dea39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85330014a69f449db1714b216c41f852)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:18.947502 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb365462-3dc1-49f5-a1d7-278e794dea39 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:19.009185 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:39:19.233643 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-cb365462-3dc1-49f5-a1d7-278e794dea39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85330014a69f449db1714b216c41f852'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:19.234455 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-cb365462-3dc1-49f5-a1d7-278e794dea39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85330014a69f449db1714b216c41f852'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:19.235214 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-cb365462-3dc1-49f5-a1d7-278e794dea39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85330014a69f449db1714b216c41f852'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:19.245357 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-cb365462-3dc1-49f5-a1d7-278e794dea39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85330014a69f449db1714b216c41f852'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:19.262176 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-cb365462-3dc1-49f5-a1d7-278e794dea39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85330014a69f449db1714b216c41f852'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:19.386374 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 843/1660] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:39:18 2022] DELETE /identity/v3/users/85330014a69f449db1714b216c41f852 => generated 0 bytes in 709 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:39:20.589281 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c7e469-539a-428e-a26e-519a2c2f44d6 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:20.590766 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-88c7e469-539a-428e-a26e-519a2c2f44d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:20.622806 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c7e469-539a-428e-a26e-519a2c2f44d6 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:20.737508 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:39:20.774721 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643072c5-9493-4885-8482-115ca2e99d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:20.778850 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643072c5-9493-4885-8482-115ca2e99d47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:20.779014 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643072c5-9493-4885-8482-115ca2e99d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:20.779344 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643072c5-9493-4885-8482-115ca2e99d47 admin admin] PATH_INFO: `/v3/projects/f26f06142ceb45088bb0c1b74aee4dc0` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:20.785220 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643072c5-9493-4885-8482-115ca2e99d47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f26f06142ceb45088bb0c1b74aee4dc0)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:20.790412 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643072c5-9493-4885-8482-115ca2e99d47 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:20.988897 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-643072c5-9493-4885-8482-115ca2e99d47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f26f06142ceb45088bb0c1b74aee4dc0', 'request_id': 'req-643072c5-9493-4885-8482-115ca2e99d47'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:21.118643 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-643072c5-9493-4885-8482-115ca2e99d47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f26f06142ceb45088bb0c1b74aee4dc0', 'request_id': 'req-643072c5-9493-4885-8482-115ca2e99d47'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:21.130352 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-643072c5-9493-4885-8482-115ca2e99d47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f26f06142ceb45088bb0c1b74aee4dc0', 'request_id': 'req-643072c5-9493-4885-8482-115ca2e99d47'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:21.159916 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-643072c5-9493-4885-8482-115ca2e99d47 admin admin] The token cache is being invalidate because project f26f06142ceb45088bb0c1b74aee4dc0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:39:21.174879 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-643072c5-9493-4885-8482-115ca2e99d47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:21.180970 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-643072c5-9493-4885-8482-115ca2e99d47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:21.180970 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 818/1661] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:39:20 2022] DELETE /identity/v3/projects/f26f06142ceb45088bb0c1b74aee4dc0 => generated 0 bytes in 590 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:39:21.182188 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb365462-3dc1-49f5-a1d7-278e794dea39 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:21.183875 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-cb365462-3dc1-49f5-a1d7-278e794dea39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:21.213580 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb365462-3dc1-49f5-a1d7-278e794dea39 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:21.329845 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e2da12-360c-4941-ba9d-3e48a145f43a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:21.332196 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e2da12-360c-4941-ba9d-3e48a145f43a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:21.338905 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e2da12-360c-4941-ba9d-3e48a145f43a admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:21.339340 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e2da12-360c-4941-ba9d-3e48a145f43a admin admin] PATH_INFO: `/v3/users/3b50ca1c44764618ac031785d365038a` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:21.341069 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e2da12-360c-4941-ba9d-3e48a145f43a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b50ca1c44764618ac031785d365038a)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:21.343400 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e2da12-360c-4941-ba9d-3e48a145f43a admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:21.584255 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643072c5-9493-4885-8482-115ca2e99d47 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:21.586691 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-643072c5-9493-4885-8482-115ca2e99d47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:21.647479 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643072c5-9493-4885-8482-115ca2e99d47 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:21.650286 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-b7e2da12-360c-4941-ba9d-3e48a145f43a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b50ca1c44764618ac031785d365038a'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:21.650986 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-b7e2da12-360c-4941-ba9d-3e48a145f43a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b50ca1c44764618ac031785d365038a'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:21.652193 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-b7e2da12-360c-4941-ba9d-3e48a145f43a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b50ca1c44764618ac031785d365038a'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:21.664735 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fc8e93-deb0-41f6-8390-908dac1c5f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:21.665638 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fc8e93-deb0-41f6-8390-908dac1c5f76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:21.665926 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fc8e93-deb0-41f6-8390-908dac1c5f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:21.666972 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fc8e93-deb0-41f6-8390-908dac1c5f76 admin admin] PATH_INFO: `/v3/projects/d547ba75dd9542368772362f9f13e72d` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:21.674132 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fc8e93-deb0-41f6-8390-908dac1c5f76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d547ba75dd9542368772362f9f13e72d)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:21.674966 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-b7e2da12-360c-4941-ba9d-3e48a145f43a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b50ca1c44764618ac031785d365038a'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:21.679052 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fc8e93-deb0-41f6-8390-908dac1c5f76 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:21.710246 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-b7e2da12-360c-4941-ba9d-3e48a145f43a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b50ca1c44764618ac031785d365038a'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:21.800139 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 844/1662] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:39:21 2022] DELETE /identity/v3/users/3b50ca1c44764618ac031785d365038a => generated 0 bytes in 619 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:39:21.862763 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-69fc8e93-deb0-41f6-8390-908dac1c5f76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd547ba75dd9542368772362f9f13e72d', 'request_id': 'req-69fc8e93-deb0-41f6-8390-908dac1c5f76'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:21.941446 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-69fc8e93-deb0-41f6-8390-908dac1c5f76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd547ba75dd9542368772362f9f13e72d', 'request_id': 'req-69fc8e93-deb0-41f6-8390-908dac1c5f76'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:21.949970 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-69fc8e93-deb0-41f6-8390-908dac1c5f76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd547ba75dd9542368772362f9f13e72d', 'request_id': 'req-69fc8e93-deb0-41f6-8390-908dac1c5f76'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:21.979766 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-69fc8e93-deb0-41f6-8390-908dac1c5f76 admin admin] The token cache is being invalidate because project d547ba75dd9542368772362f9f13e72d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:39:21.980312 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-69fc8e93-deb0-41f6-8390-908dac1c5f76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:21.983064 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-69fc8e93-deb0-41f6-8390-908dac1c5f76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:21.985161 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 819/1663] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:39:21 2022] DELETE /identity/v3/projects/d547ba75dd9542368772362f9f13e72d => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:39:21.990855 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d379a41-8d69-4a8e-8d6f-1ba890e5f2c0 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:21.991152 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d379a41-8d69-4a8e-8d6f-1ba890e5f2c0 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:21.991456 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d379a41-8d69-4a8e-8d6f-1ba890e5f2c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:22.034003 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-7d379a41-8d69-4a8e-8d6f-1ba890e5f2c0 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:22.035255 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-7d379a41-8d69-4a8e-8d6f-1ba890e5f2c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:22.061102 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:39:22.208299 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f624c084-c621-47b3-8427-33c055bf975e None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:22.208662 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f624c084-c621-47b3-8427-33c055bf975e None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:22.211180 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f624c084-c621-47b3-8427-33c055bf975e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:22.219189 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 845/1664] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:39:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:39:22.268051 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-f624c084-c621-47b3-8427-33c055bf975e None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:22.269531 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-f624c084-c621-47b3-8427-33c055bf975e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:22.307776 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d379a41-8d69-4a8e-8d6f-1ba890e5f2c0 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:22.320151 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 820/1665] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:39:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:39:22.323698 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d379a41-8d69-4a8e-8d6f-1ba890e5f2c0 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:22.366003 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aec9373-3da1-4003-84f1-2818e49c0a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:22.366910 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec9373-3da1-4003-84f1-2818e49c0a6f admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:22.367329 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec9373-3da1-4003-84f1-2818e49c0a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:22.368485 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec9373-3da1-4003-84f1-2818e49c0a6f admin admin] PATH_INFO: `/v3/domains` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:22.369407 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aec9373-3da1-4003-84f1-2818e49c0a6f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:39:22.370597 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aec9373-3da1-4003-84f1-2818e49c0a6f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:22.372419 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aec9373-3da1-4003-84f1-2818e49c0a6f admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:22.395656 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f624c084-c621-47b3-8427-33c055bf975e None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:22.407274 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f624c084-c621-47b3-8427-33c055bf975e None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:22.408416 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 846/1666] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jun 7 14:39:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:39:22.421023 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f11131-b6a5-4474-bcda-57fbff54bacc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:22.422338 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f11131-b6a5-4474-bcda-57fbff54bacc admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:22.422624 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f11131-b6a5-4474-bcda-57fbff54bacc admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:22.422953 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f11131-b6a5-4474-bcda-57fbff54bacc admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:22.423713 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f11131-b6a5-4474-bcda-57fbff54bacc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:22.433689 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f11131-b6a5-4474-bcda-57fbff54bacc admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:22.613484 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 821/1667] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:39:22 2022] POST /identity/v3/projects => generated 403 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:39:22.618294 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aec9373-3da1-4003-84f1-2818e49c0a6f admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:22.632010 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aec9373-3da1-4003-84f1-2818e49c0a6f admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:22.772425 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600ad9c0-30c6-4ab9-a28a-414bfee73118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:22.772425 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600ad9c0-30c6-4ab9-a28a-414bfee73118 admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:22.772425 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600ad9c0-30c6-4ab9-a28a-414bfee73118 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:22.772425 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600ad9c0-30c6-4ab9-a28a-414bfee73118 admin admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:22.772737 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600ad9c0-30c6-4ab9-a28a-414bfee73118 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:22.772737 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600ad9c0-30c6-4ab9-a28a-414bfee73118 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:22.864565 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f11131-b6a5-4474-bcda-57fbff54bacc admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:22.866865 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-97f11131-b6a5-4474-bcda-57fbff54bacc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:22.892100 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f11131-b6a5-4474-bcda-57fbff54bacc admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:22.903010 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 847/1668] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:39:22 2022] POST /identity/v3/users => generated 302 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:39:22.908759 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600ad9c0-30c6-4ab9-a28a-414bfee73118 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:22.913645 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d7218f-0ce2-495a-92fb-e61059516eb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:22.914747 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d7218f-0ce2-495a-92fb-e61059516eb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:22.917335 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d7218f-0ce2-495a-92fb-e61059516eb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:22.917895 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d7218f-0ce2-495a-92fb-e61059516eb0 admin admin] PATH_INFO: `/v3/projects/a03392d5ef2a4962b4513dbcbf93693f` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:22.928225 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600ad9c0-30c6-4ab9-a28a-414bfee73118 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:22.928952 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d7218f-0ce2-495a-92fb-e61059516eb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a03392d5ef2a4962b4513dbcbf93693f)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:22.931988 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d7218f-0ce2-495a-92fb-e61059516eb0 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:22.947993 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7150ace2-44ee-4d31-bcf3-81da1c1cd7fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:22.949474 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7150ace2-44ee-4d31-bcf3-81da1c1cd7fd admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:22.949778 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7150ace2-44ee-4d31-bcf3-81da1c1cd7fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:22.950654 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7150ace2-44ee-4d31-bcf3-81da1c1cd7fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:22.951903 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7150ace2-44ee-4d31-bcf3-81da1c1cd7fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:22.956359 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7150ace2-44ee-4d31-bcf3-81da1c1cd7fd admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:22.982756 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 848/1669] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:39:22 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 14:39:22.987408 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7150ace2-44ee-4d31-bcf3-81da1c1cd7fd admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:23.007057 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7150ace2-44ee-4d31-bcf3-81da1c1cd7fd admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:23.021781 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4ec087-e3bf-4992-bf38-1c44d9c44f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:23.023394 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4ec087-e3bf-4992-bf38-1c44d9c44f58 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:23.023825 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4ec087-e3bf-4992-bf38-1c44d9c44f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:23.024234 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4ec087-e3bf-4992-bf38-1c44d9c44f58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:23.025176 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4ec087-e3bf-4992-bf38-1c44d9c44f58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:23.028161 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4ec087-e3bf-4992-bf38-1c44d9c44f58 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:23.038458 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=81,keystone:DELETE=16,keystone:INSERT=3 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:39:23.045235 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 849/1670] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:39:22 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:39:23.049357 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4ec087-e3bf-4992-bf38-1c44d9c44f58 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:23.067430 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4ec087-e3bf-4992-bf38-1c44d9c44f58 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:23.068502 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-39d7218f-0ce2-495a-92fb-e61059516eb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a03392d5ef2a4962b4513dbcbf93693f', 'request_id': 'req-39d7218f-0ce2-495a-92fb-e61059516eb0'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:23.184356 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5109ee-b562-4ace-beec-db2db2c6c39a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:23.185491 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5109ee-b562-4ace-beec-db2db2c6c39a admin admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:23.185917 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5109ee-b562-4ace-beec-db2db2c6c39a admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:23.186488 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5109ee-b562-4ace-beec-db2db2c6c39a admin admin] PATH_INFO: `/v3/projects/88e4de5957bd439486627366692a8b79/users/e7d75fce4d694784b5f1c22a491cd79f/roles/3771341b3be641139e752aab690397a4` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:23.192866 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-39d7218f-0ce2-495a-92fb-e61059516eb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a03392d5ef2a4962b4513dbcbf93693f', 'request_id': 'req-39d7218f-0ce2-495a-92fb-e61059516eb0'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:23.200035 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-39d7218f-0ce2-495a-92fb-e61059516eb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a03392d5ef2a4962b4513dbcbf93693f', 'request_id': 'req-39d7218f-0ce2-495a-92fb-e61059516eb0'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:23.230529 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5109ee-b562-4ace-beec-db2db2c6c39a admin admin] RBAC: Authorizing `identity:create_grant(project_id=88e4de5957bd439486627366692a8b79, user_id=e7d75fce4d694784b5f1c22a491cd79f, role_id=3771341b3be641139e752aab690397a4)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:23.234345 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-39d7218f-0ce2-495a-92fb-e61059516eb0 admin admin] The token cache is being invalidate because project a03392d5ef2a4962b4513dbcbf93693f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:39:23.234654 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-39d7218f-0ce2-495a-92fb-e61059516eb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:23.234849 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5109ee-b562-4ace-beec-db2db2c6c39a admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:23.237706 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-39d7218f-0ce2-495a-92fb-e61059516eb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:23.240693 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 822/1671] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:39:22 2022] DELETE /identity/v3/projects/a03392d5ef2a4962b4513dbcbf93693f => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:39:23.251498 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20b874c-b15a-42b2-a0bf-0c0bf92f9e55 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:23.253746 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20b874c-b15a-42b2-a0bf-0c0bf92f9e55 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:23.258140 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20b874c-b15a-42b2-a0bf-0c0bf92f9e55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:23.283492 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-b20b874c-b15a-42b2-a0bf-0c0bf92f9e55 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:23.284789 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-b20b874c-b15a-42b2-a0bf-0c0bf92f9e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:23.298477 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 823/1672] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:39:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:39:23.307361 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5282335d-ed52-40e3-99f9-21fdedfa0677 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:23.307655 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5282335d-ed52-40e3-99f9-21fdedfa0677 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:23.307943 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5282335d-ed52-40e3-99f9-21fdedfa0677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:23.334196 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-5282335d-ed52-40e3-99f9-21fdedfa0677 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:23.335462 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-5282335d-ed52-40e3-99f9-21fdedfa0677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:23.359688 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:39:23.370586 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 850/1673] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:39:23 2022] PUT /identity/v3/projects/88e4de5957bd439486627366692a8b79/users/e7d75fce4d694784b5f1c22a491cd79f/roles/3771341b3be641139e752aab690397a4 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:39:23.377979 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f6faf5-dcd6-4807-aa6d-5c750a1760b7 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:23.378485 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f6faf5-dcd6-4807-aa6d-5c750a1760b7 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:23.378955 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f6faf5-dcd6-4807-aa6d-5c750a1760b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:23.400307 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 824/1674] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:39:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:39:23.404785 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5282335d-ed52-40e3-99f9-21fdedfa0677 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:23.427326 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5282335d-ed52-40e3-99f9-21fdedfa0677 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:23.455681 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-c6f6faf5-dcd6-4807-aa6d-5c750a1760b7 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:23.457471 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-c6f6faf5-dcd6-4807-aa6d-5c750a1760b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:23.565750 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20a317e-32c5-4ec2-b591-e2ff4cc87052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:23.566444 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20a317e-32c5-4ec2-b591-e2ff4cc87052 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:23.566729 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20a317e-32c5-4ec2-b591-e2ff4cc87052 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:23.567003 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20a317e-32c5-4ec2-b591-e2ff4cc87052 admin admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:23.568113 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20a317e-32c5-4ec2-b591-e2ff4cc87052 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:39:23.568336 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20a317e-32c5-4ec2-b591-e2ff4cc87052 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:23.574025 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20a317e-32c5-4ec2-b591-e2ff4cc87052 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:23.604703 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 825/1675] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jun 7 14:39:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:39:23.607003 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20a317e-32c5-4ec2-b591-e2ff4cc87052 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:23.611489 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 851/1676] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:39:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:39:23.630960 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20a317e-32c5-4ec2-b591-e2ff4cc87052 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:23.644107 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aedefd-beed-4f80-987f-bba370ac3fea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:23.645265 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aedefd-beed-4f80-987f-bba370ac3fea admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:23.645690 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aedefd-beed-4f80-987f-bba370ac3fea admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:23.646103 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aedefd-beed-4f80-987f-bba370ac3fea admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:23.648905 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39aedefd-beed-4f80-987f-bba370ac3fea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:23.651201 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39aedefd-beed-4f80-987f-bba370ac3fea admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:23.686410 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f6faf5-dcd6-4807-aa6d-5c750a1760b7 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:23.687767 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-c6f6faf5-dcd6-4807-aa6d-5c750a1760b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:23.773021 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f6faf5-dcd6-4807-aa6d-5c750a1760b7 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:23.778863 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d6ce1a-bbc6-4bbd-bbe8-3095d38ecff6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:23.780067 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d6ce1a-bbc6-4bbd-bbe8-3095d38ecff6 service neutron] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:23.781209 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d6ce1a-bbc6-4bbd-bbe8-3095d38ecff6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:23.781765 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d6ce1a-bbc6-4bbd-bbe8-3095d38ecff6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:23.806462 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d6ce1a-bbc6-4bbd-bbe8-3095d38ecff6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:23.808931 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d6ce1a-bbc6-4bbd-bbe8-3095d38ecff6 service neutron] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:23.832855 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 852/1677] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:39:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 14:39:23.850776 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 826/1678] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:39:23 2022] POST /identity/v3/projects => generated 365 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:39:23.851536 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d6ce1a-bbc6-4bbd-bbe8-3095d38ecff6 service neutron] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:23.864102 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d6ce1a-bbc6-4bbd-bbe8-3095d38ecff6 service neutron] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:24.049749 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fe65e5-701f-4646-97d3-551a32a22bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:24.050672 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fe65e5-701f-4646-97d3-551a32a22bf4 admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:24.051570 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fe65e5-701f-4646-97d3-551a32a22bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:24.051570 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fe65e5-701f-4646-97d3-551a32a22bf4 admin admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:24.053152 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fe65e5-701f-4646-97d3-551a32a22bf4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:24.056108 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fe65e5-701f-4646-97d3-551a32a22bf4 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:24.195070 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 853/1679] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:39:23 2022] POST /identity/v3/users => generated 283 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:39:24.200011 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aedefd-beed-4f80-987f-bba370ac3fea admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:24.224168 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aedefd-beed-4f80-987f-bba370ac3fea admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:24.247433 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc2577f-a1c9-4ecf-8ce1-6ccaee6eb848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:24.248696 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc2577f-a1c9-4ecf-8ce1-6ccaee6eb848 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:24.253361 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc2577f-a1c9-4ecf-8ce1-6ccaee6eb848 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:24.253789 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc2577f-a1c9-4ecf-8ce1-6ccaee6eb848 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:24.254704 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc2577f-a1c9-4ecf-8ce1-6ccaee6eb848 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:24.264073 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc2577f-a1c9-4ecf-8ce1-6ccaee6eb848 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:24.288417 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 827/1680] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:39:24 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 14:39:24.289612 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fe65e5-701f-4646-97d3-551a32a22bf4 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:24.328164 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fe65e5-701f-4646-97d3-551a32a22bf4 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:24.347768 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd66f1d4-d91a-4840-932d-823e4113cc26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:24.349512 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd66f1d4-d91a-4840-932d-823e4113cc26 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:24.349824 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd66f1d4-d91a-4840-932d-823e4113cc26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:24.350108 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd66f1d4-d91a-4840-932d-823e4113cc26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:24.350889 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd66f1d4-d91a-4840-932d-823e4113cc26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:24.352692 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd66f1d4-d91a-4840-932d-823e4113cc26 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:24.388579 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 854/1681] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:39:24 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:39:24.390577 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc2577f-a1c9-4ecf-8ce1-6ccaee6eb848 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:24.410611 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc2577f-a1c9-4ecf-8ce1-6ccaee6eb848 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:24.421225 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c83764-aa6d-430d-bee0-a7a40e99d0ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:24.422829 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c83764-aa6d-430d-bee0-a7a40e99d0ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:24.423138 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c83764-aa6d-430d-bee0-a7a40e99d0ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:24.424996 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c83764-aa6d-430d-bee0-a7a40e99d0ca admin admin] PATH_INFO: `/v3/projects/c39871cb446c4a7c82d72754b2667e6a/users/4b110006de2d434789d1d7fda2d73c6c/roles/3771341b3be641139e752aab690397a4` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:24.453816 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c83764-aa6d-430d-bee0-a7a40e99d0ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=c39871cb446c4a7c82d72754b2667e6a, user_id=4b110006de2d434789d1d7fda2d73c6c, role_id=3771341b3be641139e752aab690397a4)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:24.456536 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c83764-aa6d-430d-bee0-a7a40e99d0ca admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:24.609347 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 828/1682] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:39:24 2022] PUT /identity/v3/projects/c39871cb446c4a7c82d72754b2667e6a/users/4b110006de2d434789d1d7fda2d73c6c/roles/3771341b3be641139e752aab690397a4 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:39:24.611198 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ae4de3-6eb9-43c5-bc16-5912fce629bc None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:24.611504 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ae4de3-6eb9-43c5-bc16-5912fce629bc None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:24.611789 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ae4de3-6eb9-43c5-bc16-5912fce629bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:24.677035 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-12ae4de3-6eb9-43c5-bc16-5912fce629bc None None] MFA Rules not processed for user `4b110006de2d434789d1d7fda2d73c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:24.678505 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-12ae4de3-6eb9-43c5-bc16-5912fce629bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:24.835261 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 855/1683] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:39:24 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:39:24.839313 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c83764-aa6d-430d-bee0-a7a40e99d0ca admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:24.986394 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c83764-aa6d-430d-bee0-a7a40e99d0ca admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:25.002237 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed8b193-9631-4cb5-81e7-323e86e79ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:25.003127 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed8b193-9631-4cb5-81e7-323e86e79ed1 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:25.003576 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed8b193-9631-4cb5-81e7-323e86e79ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:25.004022 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed8b193-9631-4cb5-81e7-323e86e79ed1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:25.005085 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed8b193-9631-4cb5-81e7-323e86e79ed1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:25.008301 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed8b193-9631-4cb5-81e7-323e86e79ed1 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:25.107488 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:39:25.156601 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 829/1684] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:39:24 2022] POST /identity/v3/projects => generated 363 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:39:25.160725 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ae4de3-6eb9-43c5-bc16-5912fce629bc None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:25.177351 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ae4de3-6eb9-43c5-bc16-5912fce629bc None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:25.187830 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:39:25.271176 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e74d0c3-1699-431e-a971-d2b8d32dc7bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:25.272074 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e74d0c3-1699-431e-a971-d2b8d32dc7bb admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:25.273281 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e74d0c3-1699-431e-a971-d2b8d32dc7bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:25.273576 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e74d0c3-1699-431e-a971-d2b8d32dc7bb admin admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:25.274407 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e74d0c3-1699-431e-a971-d2b8d32dc7bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:25.276668 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e74d0c3-1699-431e-a971-d2b8d32dc7bb admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:25.430374 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520c81bf-35a4-48c5-a0a1-ee85cd15449f None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:25.431468 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520c81bf-35a4-48c5-a0a1-ee85cd15449f None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:25.434279 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520c81bf-35a4-48c5-a0a1-ee85cd15449f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:25.456116 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 856/1685] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:39:25 2022] POST /identity/v3/users => generated 288 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:39:25.460439 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e74d0c3-1699-431e-a971-d2b8d32dc7bb admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:25.507547 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e74d0c3-1699-431e-a971-d2b8d32dc7bb admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:25.516838 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03d1961-50ae-4c4c-b271-155f944cbbec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:25.517913 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03d1961-50ae-4c4c-b271-155f944cbbec admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:25.520100 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03d1961-50ae-4c4c-b271-155f944cbbec admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:25.520938 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03d1961-50ae-4c4c-b271-155f944cbbec admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:25.537666 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03d1961-50ae-4c4c-b271-155f944cbbec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:25.546671 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03d1961-50ae-4c4c-b271-155f944cbbec admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:25.578634 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-520c81bf-35a4-48c5-a0a1-ee85cd15449f None None] MFA Rules not processed for user `4fb4b0cf5f8b4274b787f327793964f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:25.594691 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-520c81bf-35a4-48c5-a0a1-ee85cd15449f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:25.619670 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 857/1686] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:39:25 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:39:25.622790 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03d1961-50ae-4c4c-b271-155f944cbbec admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:25.691552 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03d1961-50ae-4c4c-b271-155f944cbbec admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:25.727923 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e4786c-0aaf-4fb6-a878-b6440bb6b70b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:25.733715 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e4786c-0aaf-4fb6-a878-b6440bb6b70b admin admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:25.737769 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e4786c-0aaf-4fb6-a878-b6440bb6b70b admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:25.741537 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e4786c-0aaf-4fb6-a878-b6440bb6b70b admin admin] PATH_INFO: `/v3/projects/3ac8b2d0929b496a9c183349e20910ce/users/de400d5a2f164b29980b8469a9de810f/roles/9baff2b18f2649f7b757ff2e1f63c853` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:25.826962 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e4786c-0aaf-4fb6-a878-b6440bb6b70b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ac8b2d0929b496a9c183349e20910ce, user_id=de400d5a2f164b29980b8469a9de810f, role_id=9baff2b18f2649f7b757ff2e1f63c853)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:25.840117 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e4786c-0aaf-4fb6-a878-b6440bb6b70b admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:25.940326 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 858/1687] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:39:25 2022] PUT /identity/v3/projects/3ac8b2d0929b496a9c183349e20910ce/users/de400d5a2f164b29980b8469a9de810f/roles/9baff2b18f2649f7b757ff2e1f63c853 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:39:25.944972 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e4786c-0aaf-4fb6-a878-b6440bb6b70b admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:25.958368 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 830/1688] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:39:25 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 529 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:39:25.965037 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520c81bf-35a4-48c5-a0a1-ee85cd15449f None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:25.967514 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-520c81bf-35a4-48c5-a0a1-ee85cd15449f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:26.134622 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520c81bf-35a4-48c5-a0a1-ee85cd15449f None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:26.216586 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e4786c-0aaf-4fb6-a878-b6440bb6b70b admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:26.238289 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6c80fa-a230-41a8-82ed-209e74d96fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:26.239216 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6c80fa-a230-41a8-82ed-209e74d96fd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:26.239393 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6c80fa-a230-41a8-82ed-209e74d96fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:26.239631 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6c80fa-a230-41a8-82ed-209e74d96fd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:26.243353 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6c80fa-a230-41a8-82ed-209e74d96fd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:26.247310 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6c80fa-a230-41a8-82ed-209e74d96fd6 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:26.297727 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 859/1689] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:39:25 2022] GET /identity/v3/roles => generated 1420 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 14:39:26.300632 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6c80fa-a230-41a8-82ed-209e74d96fd6 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:26.324419 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6c80fa-a230-41a8-82ed-209e74d96fd6 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:26.346673 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d31e234-2d43-490d-b827-43784c041e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:26.347676 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d31e234-2d43-490d-b827-43784c041e88 admin admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:26.347884 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d31e234-2d43-490d-b827-43784c041e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:26.350960 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d31e234-2d43-490d-b827-43784c041e88 admin admin] PATH_INFO: `/v3/projects/3ac8b2d0929b496a9c183349e20910ce/users/de400d5a2f164b29980b8469a9de810f/roles/9baff2b18f2649f7b757ff2e1f63c853` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:26.350960 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d31e234-2d43-490d-b827-43784c041e88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ac8b2d0929b496a9c183349e20910ce, user_id=de400d5a2f164b29980b8469a9de810f, role_id=9baff2b18f2649f7b757ff2e1f63c853)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:26.359402 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d31e234-2d43-490d-b827-43784c041e88 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:26.365694 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f12c443-47b0-45e1-b69f-154d78019276 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e857db4902e42989d0d040bbc8517c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:26.366460 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f12c443-47b0-45e1-b69f-154d78019276 service nova] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:26.385436 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f12c443-47b0-45e1-b69f-154d78019276 service nova] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:26.385436 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f12c443-47b0-45e1-b69f-154d78019276 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:26.407929 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 860/1690] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:39:26 2022] PUT /identity/v3/projects/3ac8b2d0929b496a9c183349e20910ce/users/de400d5a2f164b29980b8469a9de810f/roles/9baff2b18f2649f7b757ff2e1f63c853 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:39:26.414572 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a226ff37-99ab-4789-85b8-02e4ef9f3434 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:26.414572 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a226ff37-99ab-4789-85b8-02e4ef9f3434 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:26.414572 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a226ff37-99ab-4789-85b8-02e4ef9f3434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:26.484476 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-a226ff37-99ab-4789-85b8-02e4ef9f3434 None None] MFA Rules not processed for user `de400d5a2f164b29980b8469a9de810f`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:26.485711 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-a226ff37-99ab-4789-85b8-02e4ef9f3434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:26.710026 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f12c443-47b0-45e1-b69f-154d78019276 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:26.715853 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f12c443-47b0-45e1-b69f-154d78019276 service nova] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:26.766199 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 831/1691] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:39:25 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 801 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 14:39:26.767736 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b38c54d-9a8f-4d33-81c1-918a01955454 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:26.768081 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b38c54d-9a8f-4d33-81c1-918a01955454 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:26.768360 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b38c54d-9a8f-4d33-81c1-918a01955454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:26.790644 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 861/1692] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:39:26 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 382 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:39:26.804509 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a226ff37-99ab-4789-85b8-02e4ef9f3434 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:26.814282 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-a226ff37-99ab-4789-85b8-02e4ef9f3434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:26.883860 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:39:26.899306 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-5b38c54d-9a8f-4d33-81c1-918a01955454 None None] MFA Rules not processed for user `ad9d021b171244d796c320aa9f8450a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:26.904571 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-5b38c54d-9a8f-4d33-81c1-918a01955454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:27.060112 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a226ff37-99ab-4789-85b8-02e4ef9f3434 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:27.069387 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa2ea6b-e4e0-4d97-8adf-dd35b7e59738 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d81f597cb44c56a966537089bcce4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:27.070343 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa2ea6b-e4e0-4d97-8adf-dd35b7e59738 service cinder] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:27.071316 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa2ea6b-e4e0-4d97-8adf-dd35b7e59738 service cinder] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:27.071909 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa2ea6b-e4e0-4d97-8adf-dd35b7e59738 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:27.086888 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa2ea6b-e4e0-4d97-8adf-dd35b7e59738 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:27.095221 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa2ea6b-e4e0-4d97-8adf-dd35b7e59738 service cinder] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:27.104004 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 832/1693] 10.222.0.85 () {58 vars in 1009 bytes} [Tue Jun 7 14:39:26 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:39:27.107158 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 862/1694] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:39:26 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 14:39:27.474845 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b38c54d-9a8f-4d33-81c1-918a01955454 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:27.481354 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-5b38c54d-9a8f-4d33-81c1-918a01955454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:27.491789 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa2ea6b-e4e0-4d97-8adf-dd35b7e59738 service cinder] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:27.500739 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-efa2ea6b-e4e0-4d97-8adf-dd35b7e59738 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:27.516942 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa2ea6b-e4e0-4d97-8adf-dd35b7e59738 service cinder] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:27.522160 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc51c187-f880-48e8-8fc7-374a7d482ccc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d81f597cb44c56a966537089bcce4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:27.522885 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc51c187-f880-48e8-8fc7-374a7d482ccc service cinder] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:27.523183 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc51c187-f880-48e8-8fc7-374a7d482ccc service cinder] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:27.523469 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc51c187-f880-48e8-8fc7-374a7d482ccc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:27.631107 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b38c54d-9a8f-4d33-81c1-918a01955454 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:27.643515 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a5c68c-6e91-49b2-a553-d0e696e8ed5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:27.644641 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a5c68c-6e91-49b2-a553-d0e696e8ed5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:27.644948 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc51c187-f880-48e8-8fc7-374a7d482ccc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:27.645098 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a5c68c-6e91-49b2-a553-d0e696e8ed5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:27.645474 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a5c68c-6e91-49b2-a553-d0e696e8ed5b admin admin] PATH_INFO: `/v3/users/cee40d2493384405996c822271013d23` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:27.646531 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc51c187-f880-48e8-8fc7-374a7d482ccc service cinder] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:27.647534 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a5c68c-6e91-49b2-a553-d0e696e8ed5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=cee40d2493384405996c822271013d23)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:27.653143 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a5c68c-6e91-49b2-a553-d0e696e8ed5b admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:27.661151 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 863/1695] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:39:27 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:39:27.663134 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869b8c22-be9c-41af-9905-8ca756390549 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:27.663650 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869b8c22-be9c-41af-9905-8ca756390549 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:27.664081 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869b8c22-be9c-41af-9905-8ca756390549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:27.717195 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-869b8c22-be9c-41af-9905-8ca756390549 None None] MFA Rules not processed for user `34eb9ec1f3ff44bbb03cf934cb2a04de`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:27.719437 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-869b8c22-be9c-41af-9905-8ca756390549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:27.854366 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 864/1696] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:39:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:39:27.857282 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228ab598-a01f-4721-a4f8-4535c030fd18 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:27.857752 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228ab598-a01f-4721-a4f8-4535c030fd18 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:27.858208 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228ab598-a01f-4721-a4f8-4535c030fd18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:27.907004 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-228ab598-a01f-4721-a4f8-4535c030fd18 None None] MFA Rules not processed for user `48bbf88b32774d5da9ef93ac75686c34`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:27.908991 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-228ab598-a01f-4721-a4f8-4535c030fd18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:27.984294 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:39:28.020326 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 865/1697] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:39:27 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:39:28.021505 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-d4a5c68c-6e91-49b2-a553-d0e696e8ed5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cee40d2493384405996c822271013d23'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:28.021835 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228ab598-a01f-4721-a4f8-4535c030fd18 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:28.023387 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-228ab598-a01f-4721-a4f8-4535c030fd18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:28.024844 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-d4a5c68c-6e91-49b2-a553-d0e696e8ed5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cee40d2493384405996c822271013d23'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:28.025569 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-d4a5c68c-6e91-49b2-a553-d0e696e8ed5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cee40d2493384405996c822271013d23'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:28.038164 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-d4a5c68c-6e91-49b2-a553-d0e696e8ed5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cee40d2493384405996c822271013d23'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:28.055140 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-d4a5c68c-6e91-49b2-a553-d0e696e8ed5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cee40d2493384405996c822271013d23'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:28.143858 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 833/1698] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:39:27 2022] DELETE /identity/v3/users/cee40d2493384405996c822271013d23 => generated 0 bytes in 671 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:39:28.145594 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a5c68c-6e91-49b2-a553-d0e696e8ed5b admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:28.148559 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-d4a5c68c-6e91-49b2-a553-d0e696e8ed5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:28.169808 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228ab598-a01f-4721-a4f8-4535c030fd18 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:28.374227 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b43175-b325-417e-8d57-897e6c68b876 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35aa67025f66459da0c0fc3cbfd45430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:28.374937 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b43175-b325-417e-8d57-897e6c68b876 service swift] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:28.375225 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b43175-b325-417e-8d57-897e6c68b876 service swift] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:28.375503 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b43175-b325-417e-8d57-897e6c68b876 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:28.380770 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-04b43175-b325-417e-8d57-897e6c68b876 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:28.383192 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a5c68c-6e91-49b2-a553-d0e696e8ed5b admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:28.390876 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be32b0f3-fa16-4630-9d41-6eda23b9e28e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e857db4902e42989d0d040bbc8517c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:28.391668 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be32b0f3-fa16-4630-9d41-6eda23b9e28e service nova] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:28.392052 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be32b0f3-fa16-4630-9d41-6eda23b9e28e service nova] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:28.392691 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be32b0f3-fa16-4630-9d41-6eda23b9e28e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:28.394918 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-be32b0f3-fa16-4630-9d41-6eda23b9e28e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:28.505248 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b43175-b325-417e-8d57-897e6c68b876 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:28.514422 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b43175-b325-417e-8d57-897e6c68b876 service swift] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:28.535576 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be32b0f3-fa16-4630-9d41-6eda23b9e28e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:28.537505 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be32b0f3-fa16-4630-9d41-6eda23b9e28e service nova] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:28.541561 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 866/1699] 10.222.0.85 () {60 vars in 1384 bytes} [Tue Jun 7 14:39:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 14:39:28.554680 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 834/1700] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:39:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:39:28.556732 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bfda84-2706-4f33-b941-054e10ba4c74 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:28.557448 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bfda84-2706-4f33-b941-054e10ba4c74 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:28.557875 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bfda84-2706-4f33-b941-054e10ba4c74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:28.559106 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec23b1e0-3c7f-4c7e-b424-96c7f3a4a8ac None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:28.559409 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec23b1e0-3c7f-4c7e-b424-96c7f3a4a8ac None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:28.559699 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec23b1e0-3c7f-4c7e-b424-96c7f3a4a8ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:28.621593 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-15bfda84-2706-4f33-b941-054e10ba4c74 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:28.622888 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-15bfda84-2706-4f33-b941-054e10ba4c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:28.631433 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-ec23b1e0-3c7f-4c7e-b424-96c7f3a4a8ac None None] MFA Rules not processed for user `48bbf88b32774d5da9ef93ac75686c34`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:28.639580 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-ec23b1e0-3c7f-4c7e-b424-96c7f3a4a8ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:28.853306 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 867/1701] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:39:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:39:28.855498 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0703de2b-cb44-418a-930f-cfaa1d2815ee None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:28.858039 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0703de2b-cb44-418a-930f-cfaa1d2815ee None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:28.858039 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0703de2b-cb44-418a-930f-cfaa1d2815ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:28.861970 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 835/1702] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:39:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:39:28.864024 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d292c44-2395-4dfa-bf66-01f45598f22d None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:28.864321 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d292c44-2395-4dfa-bf66-01f45598f22d None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:28.867538 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d292c44-2395-4dfa-bf66-01f45598f22d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:28.927582 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-0703de2b-cb44-418a-930f-cfaa1d2815ee None None] MFA Rules not processed for user `7ac4c85918c045b793a1a53300c989a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:28.940134 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-0703de2b-cb44-418a-930f-cfaa1d2815ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:28.955718 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-7d292c44-2395-4dfa-bf66-01f45598f22d None None] MFA Rules not processed for user `e7d75fce4d694784b5f1c22a491cd79f`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:28.961328 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-7d292c44-2395-4dfa-bf66-01f45598f22d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:29.061708 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:39:29.095584 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 868/1703] 10.222.0.94 () {58 vars in 1015 bytes} [Tue Jun 7 14:39:28 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:39:29.096693 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0703de2b-cb44-418a-930f-cfaa1d2815ee None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:29.102667 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-0703de2b-cb44-418a-930f-cfaa1d2815ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:29.176508 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0703de2b-cb44-418a-930f-cfaa1d2815ee None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:29.192350 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 836/1704] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:39:28 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:39:29.194213 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d292c44-2395-4dfa-bf66-01f45598f22d None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:29.196029 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-7d292c44-2395-4dfa-bf66-01f45598f22d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:29.198840 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6458bb73-50fc-4eeb-a088-5343a9d79d1c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e857db4902e42989d0d040bbc8517c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:29.202791 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6458bb73-50fc-4eeb-a088-5343a9d79d1c service nova] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:29.203160 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6458bb73-50fc-4eeb-a088-5343a9d79d1c service nova] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:29.206447 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6458bb73-50fc-4eeb-a088-5343a9d79d1c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:29.213084 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-6458bb73-50fc-4eeb-a088-5343a9d79d1c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:29.216361 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d292c44-2395-4dfa-bf66-01f45598f22d None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:29.227016 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060e72e4-af79-4db3-a3a4-5283e41c392a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:29.228245 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060e72e4-af79-4db3-a3a4-5283e41c392a service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:29.228758 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060e72e4-af79-4db3-a3a4-5283e41c392a service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:29.229240 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060e72e4-af79-4db3-a3a4-5283e41c392a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:29.269922 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6458bb73-50fc-4eeb-a088-5343a9d79d1c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:29.271341 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060e72e4-af79-4db3-a3a4-5283e41c392a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:29.276503 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6458bb73-50fc-4eeb-a088-5343a9d79d1c service nova] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:29.280742 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060e72e4-af79-4db3-a3a4-5283e41c392a service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:29.298101 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 837/1705] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:39:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:39:29.299515 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060e72e4-af79-4db3-a3a4-5283e41c392a service neutron] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:29.300928 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-060e72e4-af79-4db3-a3a4-5283e41c392a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:29.317313 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 869/1706] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:39:29 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:39:29.321030 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060e72e4-af79-4db3-a3a4-5283e41c392a service neutron] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:29.329684 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dfd117-1048-418b-8c94-b0bf43f3a965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:29.330638 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dfd117-1048-418b-8c94-b0bf43f3a965 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:29.330984 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dfd117-1048-418b-8c94-b0bf43f3a965 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:29.331466 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dfd117-1048-418b-8c94-b0bf43f3a965 admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:29.332634 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dfd117-1048-418b-8c94-b0bf43f3a965 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:29.334864 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dfd117-1048-418b-8c94-b0bf43f3a965 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:29.520479 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 838/1707] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:39:29 2022] POST /identity/v3/projects => generated 405 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:39:29.524950 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6458bb73-50fc-4eeb-a088-5343a9d79d1c service nova] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:29.538422 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6458bb73-50fc-4eeb-a088-5343a9d79d1c service nova] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:29.701820 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dfd117-1048-418b-8c94-b0bf43f3a965 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:29.708202 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-95dfd117-1048-418b-8c94-b0bf43f3a965 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:29.767727 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc8c2f4-0e78-4200-b3b9-42ab78ca112e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:29.769513 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc8c2f4-0e78-4200-b3b9-42ab78ca112e admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:29.770074 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc8c2f4-0e78-4200-b3b9-42ab78ca112e admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:29.770282 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc8c2f4-0e78-4200-b3b9-42ab78ca112e admin admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:29.776603 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc8c2f4-0e78-4200-b3b9-42ab78ca112e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:29.784439 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc8c2f4-0e78-4200-b3b9-42ab78ca112e admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:30.129847 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 870/1708] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:39:29 2022] POST /identity/v3/users => generated 309 bytes in 429 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:39:30.129847 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc8c2f4-0e78-4200-b3b9-42ab78ca112e admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:30.132600 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dfd117-1048-418b-8c94-b0bf43f3a965 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:30.132600 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3ea554-0d8e-4c86-8538-c0976a878f27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:30.132600 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3ea554-0d8e-4c86-8538-c0976a878f27 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:30.132600 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3ea554-0d8e-4c86-8538-c0976a878f27 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:30.133455 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3ea554-0d8e-4c86-8538-c0976a878f27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:30.133542 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc8c2f4-0e78-4200-b3b9-42ab78ca112e admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:30.133542 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968ce007-1145-4c96-b936-90d40539cb15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:30.133542 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968ce007-1145-4c96-b936-90d40539cb15 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:30.133542 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968ce007-1145-4c96-b936-90d40539cb15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:30.133856 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968ce007-1145-4c96-b936-90d40539cb15 admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:30.133856 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968ce007-1145-4c96-b936-90d40539cb15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:30.133856 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968ce007-1145-4c96-b936-90d40539cb15 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:30.133856 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 871/1709] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:39:29 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:39:30.134139 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3ea554-0d8e-4c86-8538-c0976a878f27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:30.134237 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968ce007-1145-4c96-b936-90d40539cb15 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:30.134337 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3ea554-0d8e-4c86-8538-c0976a878f27 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:30.134864 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968ce007-1145-4c96-b936-90d40539cb15 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:30.134963 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 839/1710] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:39:29 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:39:30.135050 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53ebd34-2b75-4956-a2d0-49bb28706220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:30.135050 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ebd34-2b75-4956-a2d0-49bb28706220 admin admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:30.135050 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ebd34-2b75-4956-a2d0-49bb28706220 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:30.135050 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ebd34-2b75-4956-a2d0-49bb28706220 admin admin] PATH_INFO: `/v3/projects/b3ba9817298f4fc9b06bedcbbe8540dc/users/50ebe26d884f404eaff5464f199e9a2a/roles/9baff2b18f2649f7b757ff2e1f63c853` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:30.169859 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:39:30.324169 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53ebd34-2b75-4956-a2d0-49bb28706220 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3ba9817298f4fc9b06bedcbbe8540dc, user_id=50ebe26d884f404eaff5464f199e9a2a, role_id=9baff2b18f2649f7b757ff2e1f63c853)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:30.324169 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53ebd34-2b75-4956-a2d0-49bb28706220 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:30.606364 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 872/1711] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:39:30 2022] PUT /identity/v3/projects/b3ba9817298f4fc9b06bedcbbe8540dc/users/50ebe26d884f404eaff5464f199e9a2a/roles/9baff2b18f2649f7b757ff2e1f63c853 => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:39:30.610908 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3ea554-0d8e-4c86-8538-c0976a878f27 service neutron] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:30.634106 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3ea554-0d8e-4c86-8538-c0976a878f27 service neutron] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:30.885230 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69e1029-dcf1-444d-a34f-4bab56d8bf13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:30.888924 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69e1029-dcf1-444d-a34f-4bab56d8bf13 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:30.890110 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69e1029-dcf1-444d-a34f-4bab56d8bf13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:30.891156 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69e1029-dcf1-444d-a34f-4bab56d8bf13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:30.893043 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69e1029-dcf1-444d-a34f-4bab56d8bf13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:30.901167 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69e1029-dcf1-444d-a34f-4bab56d8bf13 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:30.958154 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 840/1712] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:39:30 2022] GET /identity/v3/roles => generated 1420 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:39:30.961129 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53ebd34-2b75-4956-a2d0-49bb28706220 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:30.988466 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53ebd34-2b75-4956-a2d0-49bb28706220 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:31.008926 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25309d63-db0c-4fe8-97f6-77d6fba88758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:31.008926 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25309d63-db0c-4fe8-97f6-77d6fba88758 admin admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:31.008926 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25309d63-db0c-4fe8-97f6-77d6fba88758 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:31.009515 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25309d63-db0c-4fe8-97f6-77d6fba88758 admin admin] PATH_INFO: `/v3/projects/b3ba9817298f4fc9b06bedcbbe8540dc/users/50ebe26d884f404eaff5464f199e9a2a/roles/9baff2b18f2649f7b757ff2e1f63c853` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:31.016345 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25309d63-db0c-4fe8-97f6-77d6fba88758 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3ba9817298f4fc9b06bedcbbe8540dc, user_id=50ebe26d884f404eaff5464f199e9a2a, role_id=9baff2b18f2649f7b757ff2e1f63c853)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:31.033077 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25309d63-db0c-4fe8-97f6-77d6fba88758 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:31.082918 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 873/1713] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:39:30 2022] PUT /identity/v3/projects/b3ba9817298f4fc9b06bedcbbe8540dc/users/50ebe26d884f404eaff5464f199e9a2a/roles/9baff2b18f2649f7b757ff2e1f63c853 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:39:31.252027 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:39:31.796152 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69e1029-dcf1-444d-a34f-4bab56d8bf13 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:31.798530 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-f69e1029-dcf1-444d-a34f-4bab56d8bf13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:31.839847 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69e1029-dcf1-444d-a34f-4bab56d8bf13 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:32.051805 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481c20b7-b161-403a-a2fd-7cdcc02f3ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:32.055142 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c20b7-b161-403a-a2fd-7cdcc02f3ac3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:32.059934 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c20b7-b161-403a-a2fd-7cdcc02f3ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:32.059934 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c20b7-b161-403a-a2fd-7cdcc02f3ac3 admin admin] PATH_INFO: `/v3/projects/01ea22e1fb6c4f378f9ff4b59ae609dd` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:32.059934 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481c20b7-b161-403a-a2fd-7cdcc02f3ac3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01ea22e1fb6c4f378f9ff4b59ae609dd)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:32.066413 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481c20b7-b161-403a-a2fd-7cdcc02f3ac3 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:32.079824 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:39:32.318506 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:39:32.348397 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5e9d16-f71e-43ff-933b-ed008100cc72 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:32.348723 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5e9d16-f71e-43ff-933b-ed008100cc72 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:32.349009 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5e9d16-f71e-43ff-933b-ed008100cc72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:32.427282 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-481c20b7-b161-403a-a2fd-7cdcc02f3ac3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01ea22e1fb6c4f378f9ff4b59ae609dd', 'request_id': 'req-481c20b7-b161-403a-a2fd-7cdcc02f3ac3'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:32.473693 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-ba5e9d16-f71e-43ff-933b-ed008100cc72 None None] MFA Rules not processed for user `48bbf88b32774d5da9ef93ac75686c34`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:32.480667 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-ba5e9d16-f71e-43ff-933b-ed008100cc72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:32.593867 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-481c20b7-b161-403a-a2fd-7cdcc02f3ac3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01ea22e1fb6c4f378f9ff4b59ae609dd', 'request_id': 'req-481c20b7-b161-403a-a2fd-7cdcc02f3ac3'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:32.603680 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-481c20b7-b161-403a-a2fd-7cdcc02f3ac3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01ea22e1fb6c4f378f9ff4b59ae609dd', 'request_id': 'req-481c20b7-b161-403a-a2fd-7cdcc02f3ac3'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:32.679309 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-481c20b7-b161-403a-a2fd-7cdcc02f3ac3 admin admin] The token cache is being invalidate because project 01ea22e1fb6c4f378f9ff4b59ae609dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:39:32.679562 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-481c20b7-b161-403a-a2fd-7cdcc02f3ac3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:32.681395 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-481c20b7-b161-403a-a2fd-7cdcc02f3ac3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:32.690231 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 841/1714] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:39:31 2022] DELETE /identity/v3/projects/01ea22e1fb6c4f378f9ff4b59ae609dd => generated 0 bytes in 891 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:39:32.692226 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3456ff2e-a3dc-47b4-adbb-5b492d2525b3 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:32.692556 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3456ff2e-a3dc-47b4-adbb-5b492d2525b3 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:32.692852 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3456ff2e-a3dc-47b4-adbb-5b492d2525b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:32.723408 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-3456ff2e-a3dc-47b4-adbb-5b492d2525b3 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:32.725227 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-3456ff2e-a3dc-47b4-adbb-5b492d2525b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:32.883885 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 874/1715] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:39:32 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 536 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:39:33.003450 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 842/1716] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:39:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:39:33.010849 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88577f4a-eae9-4bb5-9716-0b20e6b29fe8 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:33.017603 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88577f4a-eae9-4bb5-9716-0b20e6b29fe8 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:33.017603 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88577f4a-eae9-4bb5-9716-0b20e6b29fe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:33.068495 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-88577f4a-eae9-4bb5-9716-0b20e6b29fe8 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:33.071279 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-88577f4a-eae9-4bb5-9716-0b20e6b29fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:33.094849 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 875/1717] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:39:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:39:33.098455 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3456ff2e-a3dc-47b4-adbb-5b492d2525b3 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:33.143732 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3456ff2e-a3dc-47b4-adbb-5b492d2525b3 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:33.192335 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4cd30c-5705-4601-b922-214d2d06dc41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:33.192335 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4cd30c-5705-4601-b922-214d2d06dc41 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:33.192335 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4cd30c-5705-4601-b922-214d2d06dc41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:33.192335 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4cd30c-5705-4601-b922-214d2d06dc41 admin admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:33.192776 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4cd30c-5705-4601-b922-214d2d06dc41 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:39:33.193069 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4cd30c-5705-4601-b922-214d2d06dc41 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:33.197601 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4cd30c-5705-4601-b922-214d2d06dc41 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:33.256681 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88577f4a-eae9-4bb5-9716-0b20e6b29fe8 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:33.257405 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 843/1718] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jun 7 14:39:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:39:33.272671 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88577f4a-eae9-4bb5-9716-0b20e6b29fe8 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:33.286826 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303a425d-51fa-4969-9398-cd4f1da947f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:33.287791 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a425d-51fa-4969-9398-cd4f1da947f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:33.288719 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a425d-51fa-4969-9398-cd4f1da947f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:33.293763 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a425d-51fa-4969-9398-cd4f1da947f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:33.293763 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303a425d-51fa-4969-9398-cd4f1da947f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:33.295808 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303a425d-51fa-4969-9398-cd4f1da947f1 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:33.455081 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 876/1719] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:39:33 2022] POST /identity/v3/projects => generated 371 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:39:33.459249 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4cd30c-5705-4601-b922-214d2d06dc41 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:33.485851 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4cd30c-5705-4601-b922-214d2d06dc41 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:33.729185 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12376e3-258f-48f0-8333-105326f8dfa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:33.730192 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12376e3-258f-48f0-8333-105326f8dfa1 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:33.730561 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12376e3-258f-48f0-8333-105326f8dfa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:33.730911 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12376e3-258f-48f0-8333-105326f8dfa1 admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:33.733219 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12376e3-258f-48f0-8333-105326f8dfa1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:33.784839 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12376e3-258f-48f0-8333-105326f8dfa1 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:33.897303 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 844/1720] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:39:33 2022] POST /identity/v3/users => generated 286 bytes in 438 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:39:33.901241 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303a425d-51fa-4969-9398-cd4f1da947f1 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:33.920231 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303a425d-51fa-4969-9398-cd4f1da947f1 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:33.949177 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27674f74-8ed3-4577-9c82-9ed396d13f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:33.954132 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27674f74-8ed3-4577-9c82-9ed396d13f99 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:33.954770 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27674f74-8ed3-4577-9c82-9ed396d13f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:33.955340 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27674f74-8ed3-4577-9c82-9ed396d13f99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:33.959751 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27674f74-8ed3-4577-9c82-9ed396d13f99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:33.964134 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27674f74-8ed3-4577-9c82-9ed396d13f99 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:34.007431 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 877/1721] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:39:33 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:39:34.011764 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12376e3-258f-48f0-8333-105326f8dfa1 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:34.024668 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12376e3-258f-48f0-8333-105326f8dfa1 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:34.031120 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3ba94f-d8ee-46f9-bd76-efe92e68296f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:34.032170 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3ba94f-d8ee-46f9-bd76-efe92e68296f admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:34.036949 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3ba94f-d8ee-46f9-bd76-efe92e68296f admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:34.036949 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3ba94f-d8ee-46f9-bd76-efe92e68296f admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:34.036949 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3ba94f-d8ee-46f9-bd76-efe92e68296f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:34.036949 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3ba94f-d8ee-46f9-bd76-efe92e68296f admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:34.056053 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 845/1722] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:39:34 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:39:34.057866 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27674f74-8ed3-4577-9c82-9ed396d13f99 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:34.100994 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27674f74-8ed3-4577-9c82-9ed396d13f99 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:34.122886 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a24578a-c593-45c0-8ebb-bbe6c3513f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:34.128296 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a24578a-c593-45c0-8ebb-bbe6c3513f2a admin admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:34.128830 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a24578a-c593-45c0-8ebb-bbe6c3513f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:34.129262 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a24578a-c593-45c0-8ebb-bbe6c3513f2a admin admin] PATH_INFO: `/v3/projects/37770b1a2b4744f3baac63f128fdd77e/users/b7c5d5106f674f0d897e42f69012743b/roles/3771341b3be641139e752aab690397a4` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:34.173952 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a24578a-c593-45c0-8ebb-bbe6c3513f2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=37770b1a2b4744f3baac63f128fdd77e, user_id=b7c5d5106f674f0d897e42f69012743b, role_id=3771341b3be641139e752aab690397a4)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:34.200037 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a24578a-c593-45c0-8ebb-bbe6c3513f2a admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:34.308977 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:39:34.315340 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 878/1723] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:39:34 2022] PUT /identity/v3/projects/37770b1a2b4744f3baac63f128fdd77e/users/b7c5d5106f674f0d897e42f69012743b/roles/3771341b3be641139e752aab690397a4 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:39:34.326158 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc37543f-9f98-48cd-ad6b-72a4021eebf9 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:34.326452 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc37543f-9f98-48cd-ad6b-72a4021eebf9 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:34.326730 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc37543f-9f98-48cd-ad6b-72a4021eebf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:34.455764 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-cc37543f-9f98-48cd-ad6b-72a4021eebf9 None None] MFA Rules not processed for user `b7c5d5106f674f0d897e42f69012743b`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:34.461130 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-cc37543f-9f98-48cd-ad6b-72a4021eebf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:34.732345 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 846/1724] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:39:34 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 408 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:39:34.734959 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a24578a-c593-45c0-8ebb-bbe6c3513f2a admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:35.018069 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a24578a-c593-45c0-8ebb-bbe6c3513f2a admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:35.047433 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db522fcb-31ac-459f-86fd-14da0e169350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:35.048350 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db522fcb-31ac-459f-86fd-14da0e169350 admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:35.052827 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db522fcb-31ac-459f-86fd-14da0e169350 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:35.053343 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db522fcb-31ac-459f-86fd-14da0e169350 admin admin] PATH_INFO: `/v3/projects` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:35.054473 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db522fcb-31ac-459f-86fd-14da0e169350 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:35.056707 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db522fcb-31ac-459f-86fd-14da0e169350 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:35.228270 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 879/1725] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:39:34 2022] POST /identity/v3/projects => generated 371 bytes in 491 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:39:35.230286 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc37543f-9f98-48cd-ad6b-72a4021eebf9 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:35.258276 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc37543f-9f98-48cd-ad6b-72a4021eebf9 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:35.460074 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db522fcb-31ac-459f-86fd-14da0e169350 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:35.465231 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-db522fcb-31ac-459f-86fd-14da0e169350 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:35.540329 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16acb80-add9-49fc-870b-1d18686e2897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:35.541113 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16acb80-add9-49fc-870b-1d18686e2897 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:35.541459 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16acb80-add9-49fc-870b-1d18686e2897 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:35.541814 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16acb80-add9-49fc-870b-1d18686e2897 admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:35.542613 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16acb80-add9-49fc-870b-1d18686e2897 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:35.554435 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16acb80-add9-49fc-870b-1d18686e2897 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:35.652314 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 847/1726] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:39:35 2022] POST /identity/v3/users => generated 292 bytes in 423 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:39:35.656873 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16acb80-add9-49fc-870b-1d18686e2897 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:35.692243 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db522fcb-31ac-459f-86fd-14da0e169350 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:35.700349 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c53964-5adf-4277-87e3-f6dc48e25382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:35.700599 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16acb80-add9-49fc-870b-1d18686e2897 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:35.705366 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c53964-5adf-4277-87e3-f6dc48e25382 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:35.705366 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c53964-5adf-4277-87e3-f6dc48e25382 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:35.705366 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c53964-5adf-4277-87e3-f6dc48e25382 admin admin] PATH_INFO: `/v3/users/b7f92cefd82440ed8368e2b388a4bc4c` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:35.705366 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c53964-5adf-4277-87e3-f6dc48e25382 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7f92cefd82440ed8368e2b388a4bc4c)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:35.709055 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c53964-5adf-4277-87e3-f6dc48e25382 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:35.721596 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0373b9b-18ee-4fd4-ac97-5fdb821d16b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:35.722412 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0373b9b-18ee-4fd4-ac97-5fdb821d16b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:35.728666 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0373b9b-18ee-4fd4-ac97-5fdb821d16b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:35.728666 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0373b9b-18ee-4fd4-ac97-5fdb821d16b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:35.728666 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0373b9b-18ee-4fd4-ac97-5fdb821d16b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:35.732939 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0373b9b-18ee-4fd4-ac97-5fdb821d16b9 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:35.772318 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 848/1727] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:39:35 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:39:35.777640 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0373b9b-18ee-4fd4-ac97-5fdb821d16b9 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:35.801480 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0373b9b-18ee-4fd4-ac97-5fdb821d16b9 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:35.818523 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:39:35.822755 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa554fc7-4950-4862-a013-c0f302091068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:35.824299 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa554fc7-4950-4862-a013-c0f302091068 admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:35.824652 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa554fc7-4950-4862-a013-c0f302091068 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:35.824997 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa554fc7-4950-4862-a013-c0f302091068 admin admin] PATH_INFO: `/v3/projects/4617320ceb90448fa51fe6f46006730b/users/ba5fc1b1ca2b4737b15c486f0b8a63fd/roles/9baff2b18f2649f7b757ff2e1f63c853` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:35.897904 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa554fc7-4950-4862-a013-c0f302091068 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4617320ceb90448fa51fe6f46006730b, user_id=ba5fc1b1ca2b4737b15c486f0b8a63fd, role_id=9baff2b18f2649f7b757ff2e1f63c853)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:35.915719 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa554fc7-4950-4862-a013-c0f302091068 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:36.069505 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-42c53964-5adf-4277-87e3-f6dc48e25382 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7f92cefd82440ed8368e2b388a4bc4c'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:36.070926 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-42c53964-5adf-4277-87e3-f6dc48e25382 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7f92cefd82440ed8368e2b388a4bc4c'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:36.072289 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-42c53964-5adf-4277-87e3-f6dc48e25382 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7f92cefd82440ed8368e2b388a4bc4c'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:36.085053 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-42c53964-5adf-4277-87e3-f6dc48e25382 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7f92cefd82440ed8368e2b388a4bc4c'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:36.102674 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-42c53964-5adf-4277-87e3-f6dc48e25382 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7f92cefd82440ed8368e2b388a4bc4c'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:36.189304 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 849/1728] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:39:35 2022] PUT /identity/v3/projects/4617320ceb90448fa51fe6f46006730b/users/ba5fc1b1ca2b4737b15c486f0b8a63fd/roles/9baff2b18f2649f7b757ff2e1f63c853 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:39:36.193596 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa554fc7-4950-4862-a013-c0f302091068 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:36.201711 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-fa554fc7-4950-4862-a013-c0f302091068 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:36.290390 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 880/1729] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:39:35 2022] DELETE /identity/v3/users/b7f92cefd82440ed8368e2b388a4bc4c => generated 0 bytes in 810 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:39:36.292726 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e923c25a-18cf-474a-b451-fe5c17f683eb None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:36.293190 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e923c25a-18cf-474a-b451-fe5c17f683eb None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:36.301735 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e923c25a-18cf-474a-b451-fe5c17f683eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:36.333546 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:39:36.378751 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-e923c25a-18cf-474a-b451-fe5c17f683eb None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:36.385596 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-e923c25a-18cf-474a-b451-fe5c17f683eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:36.502226 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa554fc7-4950-4862-a013-c0f302091068 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:36.574555 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 881/1730] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:39:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:39:36.581359 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e923c25a-18cf-474a-b451-fe5c17f683eb None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:36.585072 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-e923c25a-18cf-474a-b451-fe5c17f683eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:36.684533 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ea8d6a-c6e2-4d7a-9a9b-7dd6146ac285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:36.689836 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ea8d6a-c6e2-4d7a-9a9b-7dd6146ac285 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:36.689836 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ea8d6a-c6e2-4d7a-9a9b-7dd6146ac285 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:36.689836 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ea8d6a-c6e2-4d7a-9a9b-7dd6146ac285 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:36.689836 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ea8d6a-c6e2-4d7a-9a9b-7dd6146ac285 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:36.690733 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ea8d6a-c6e2-4d7a-9a9b-7dd6146ac285 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:36.729009 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 850/1731] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:39:36 2022] GET /identity/v3/roles => generated 1420 bytes in 536 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:39:36.730420 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6763f97-5af7-4250-aa94-1603805acefb None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:36.730720 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6763f97-5af7-4250-aa94-1603805acefb None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:36.733733 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e923c25a-18cf-474a-b451-fe5c17f683eb None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:36.735768 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6763f97-5af7-4250-aa94-1603805acefb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:36.751212 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b739e581-16a3-48e8-a4b7-1e1a59464fa2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:36.751900 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b739e581-16a3-48e8-a4b7-1e1a59464fa2 service neutron] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:36.752290 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b739e581-16a3-48e8-a4b7-1e1a59464fa2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:36.752622 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b739e581-16a3-48e8-a4b7-1e1a59464fa2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:36.798797 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b739e581-16a3-48e8-a4b7-1e1a59464fa2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:36.805559 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b739e581-16a3-48e8-a4b7-1e1a59464fa2 service neutron] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:36.846400 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-b6763f97-5af7-4250-aa94-1603805acefb None None] MFA Rules not processed for user `50ebe26d884f404eaff5464f199e9a2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:36.847654 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-b6763f97-5af7-4250-aa94-1603805acefb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:36.851698 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 882/1732] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:39:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 14:39:36.871346 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b739e581-16a3-48e8-a4b7-1e1a59464fa2 service neutron] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:36.933581 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b739e581-16a3-48e8-a4b7-1e1a59464fa2 service neutron] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:36.984105 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd16775-c197-4dc0-82ee-0a827b1185d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:36.989350 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd16775-c197-4dc0-82ee-0a827b1185d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:36.989640 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd16775-c197-4dc0-82ee-0a827b1185d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:36.990126 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd16775-c197-4dc0-82ee-0a827b1185d0 admin admin] PATH_INFO: `/v3/projects/4617320ceb90448fa51fe6f46006730b/users/ba5fc1b1ca2b4737b15c486f0b8a63fd/roles/9baff2b18f2649f7b757ff2e1f63c853` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:36.994206 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd16775-c197-4dc0-82ee-0a827b1185d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4617320ceb90448fa51fe6f46006730b, user_id=ba5fc1b1ca2b4737b15c486f0b8a63fd, role_id=9baff2b18f2649f7b757ff2e1f63c853)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:37.002555 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd16775-c197-4dc0-82ee-0a827b1185d0 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:37.011450 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 851/1733] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:39:36 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:39:37.018095 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb520415-2175-40ca-948c-1f2098fccaa0 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:37.018393 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb520415-2175-40ca-948c-1f2098fccaa0 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:37.020039 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb520415-2175-40ca-948c-1f2098fccaa0 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:37.026661 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 852/1734] 10.222.0.85 () {54 vars in 921 bytes} [Tue Jun 7 14:39:37 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:39:37.050735 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 883/1735] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:39:36 2022] PUT /identity/v3/projects/4617320ceb90448fa51fe6f46006730b/users/ba5fc1b1ca2b4737b15c486f0b8a63fd/roles/9baff2b18f2649f7b757ff2e1f63c853 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:39:37.052214 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696bc255-4e8b-4bb3-b87f-b2df87353bfd None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:37.052540 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696bc255-4e8b-4bb3-b87f-b2df87353bfd None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:37.052855 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696bc255-4e8b-4bb3-b87f-b2df87353bfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:37.058099 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61523746-701d-4641-b5b9-617e35672f0a None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:37.059296 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61523746-701d-4641-b5b9-617e35672f0a None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:37.059902 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61523746-701d-4641-b5b9-617e35672f0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:37.089469 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-61523746-701d-4641-b5b9-617e35672f0a None None] MFA Rules not processed for user `8e857db4902e42989d0d040bbc8517c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:37.090901 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-61523746-701d-4641-b5b9-617e35672f0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:37.165967 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-696bc255-4e8b-4bb3-b87f-b2df87353bfd None None] MFA Rules not processed for user `ba5fc1b1ca2b4737b15c486f0b8a63fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:37.169205 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-696bc255-4e8b-4bb3-b87f-b2df87353bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:37.292915 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:39:37.346918 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 853/1736] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:39:37 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:39:37.354828 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 884/1737] 10.222.0.85 () {58 vars in 1018 bytes} [Tue Jun 7 14:39:37 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:39:37.357472 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696bc255-4e8b-4bb3-b87f-b2df87353bfd None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:37.360598 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-696bc255-4e8b-4bb3-b87f-b2df87353bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:37.382140 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61523746-701d-4641-b5b9-617e35672f0a None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:37.383733 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-61523746-701d-4641-b5b9-617e35672f0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:37.434937 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61523746-701d-4641-b5b9-617e35672f0a None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:37.451392 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897e3eb6-359e-45c2-8d34-9683830ecd1b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e857db4902e42989d0d040bbc8517c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:37.452129 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e3eb6-359e-45c2-8d34-9683830ecd1b service nova] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:37.452410 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e3eb6-359e-45c2-8d34-9683830ecd1b service nova] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:37.452693 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e3eb6-359e-45c2-8d34-9683830ecd1b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:37.477512 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897e3eb6-359e-45c2-8d34-9683830ecd1b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:37.480085 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897e3eb6-359e-45c2-8d34-9683830ecd1b service nova] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:37.495854 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 885/1738] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:39:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 14:39:37.520885 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696bc255-4e8b-4bb3-b87f-b2df87353bfd None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:37.531469 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ff97b9-b60c-439c-b3e5-070a30aee829 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d81f597cb44c56a966537089bcce4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:37.532253 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ff97b9-b60c-439c-b3e5-070a30aee829 service cinder] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:37.534815 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ff97b9-b60c-439c-b3e5-070a30aee829 service cinder] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:37.535126 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ff97b9-b60c-439c-b3e5-070a30aee829 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:37.569149 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ff97b9-b60c-439c-b3e5-070a30aee829 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:37.571692 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ff97b9-b60c-439c-b3e5-070a30aee829 service cinder] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:37.612579 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 854/1739] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:39:37 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:39:37.875821 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b58b8b8-ca37-4810-84ed-0e541b4636b0 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:37.876228 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b58b8b8-ca37-4810-84ed-0e541b4636b0 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:37.876690 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b58b8b8-ca37-4810-84ed-0e541b4636b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:37.914956 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-6b58b8b8-ca37-4810-84ed-0e541b4636b0 None None] MFA Rules not processed for user `4fb4b0cf5f8b4274b787f327793964f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:37.931738 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-6b58b8b8-ca37-4810-84ed-0e541b4636b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:37.990209 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:39:38.112616 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 886/1740] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:39:37 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:39:38.295458 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50437227-5852-42e5-b751-421438dd58c6 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:38.296021 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50437227-5852-42e5-b751-421438dd58c6 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:38.296513 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50437227-5852-42e5-b751-421438dd58c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:38.339977 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-50437227-5852-42e5-b751-421438dd58c6 None None] MFA Rules not processed for user `34eb9ec1f3ff44bbb03cf934cb2a04de`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:38.341893 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-50437227-5852-42e5-b751-421438dd58c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:38.369688 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b58b8b8-ca37-4810-84ed-0e541b4636b0 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:38.372336 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-6b58b8b8-ca37-4810-84ed-0e541b4636b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:38.496999 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 855/1741] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:39:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:39:38.633342 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b58b8b8-ca37-4810-84ed-0e541b4636b0 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:38.642911 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2202793-3a89-40d1-a59e-f32a9de0c191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:38.644150 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2202793-3a89-40d1-a59e-f32a9de0c191 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:38.644575 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2202793-3a89-40d1-a59e-f32a9de0c191 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:38.644994 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2202793-3a89-40d1-a59e-f32a9de0c191 admin admin] PATH_INFO: `/v3/projects/d8b2399590184f02b899e7bc739bc466` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:38.646419 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2202793-3a89-40d1-a59e-f32a9de0c191 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8b2399590184f02b899e7bc739bc466)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:38.647560 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: WARNING py.warnings [None req-d2202793-3a89-40d1-a59e-f32a9de0c191 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 14:39:38.647560 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: warnings.warn(msg) Jun 07 14:39:38.647560 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: Jun 07 14:39:38.649233 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2202793-3a89-40d1-a59e-f32a9de0c191 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:38.823269 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-d2202793-3a89-40d1-a59e-f32a9de0c191 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8b2399590184f02b899e7bc739bc466', 'request_id': 'req-d2202793-3a89-40d1-a59e-f32a9de0c191'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:38.861830 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:39:38.883502 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-d2202793-3a89-40d1-a59e-f32a9de0c191 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8b2399590184f02b899e7bc739bc466', 'request_id': 'req-d2202793-3a89-40d1-a59e-f32a9de0c191'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:38.891327 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-d2202793-3a89-40d1-a59e-f32a9de0c191 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8b2399590184f02b899e7bc739bc466', 'request_id': 'req-d2202793-3a89-40d1-a59e-f32a9de0c191'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:38.926617 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-d2202793-3a89-40d1-a59e-f32a9de0c191 admin admin] The token cache is being invalidate because project d8b2399590184f02b899e7bc739bc466 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:39:38.927154 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-d2202793-3a89-40d1-a59e-f32a9de0c191 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:38.928246 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-d2202793-3a89-40d1-a59e-f32a9de0c191 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:38.930288 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 887/1742] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:39:38 2022] DELETE /identity/v3/projects/d8b2399590184f02b899e7bc739bc466 => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:39:38.934708 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd06d654-99d3-463e-9bbb-47375d398fce None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:38.935235 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd06d654-99d3-463e-9bbb-47375d398fce None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:38.935667 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd06d654-99d3-463e-9bbb-47375d398fce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:38.968920 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-bd06d654-99d3-463e-9bbb-47375d398fce None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:38.970184 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-bd06d654-99d3-463e-9bbb-47375d398fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:39.032626 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2202793-3a89-40d1-a59e-f32a9de0c191 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:39.034637 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-d2202793-3a89-40d1-a59e-f32a9de0c191 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:39.060077 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 856/1743] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:39:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:39:39.062805 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513444b3-0064-4218-b04c-10b64101b3e2 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:39.063457 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513444b3-0064-4218-b04c-10b64101b3e2 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:39.064170 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513444b3-0064-4218-b04c-10b64101b3e2 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:39.066272 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 857/1744] 10.222.0.85 () {54 vars in 912 bytes} [Tue Jun 7 14:39:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:39:39.068043 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18d9d94-eb94-4afd-86d6-0e8e7bf981a0 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:39.068741 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18d9d94-eb94-4afd-86d6-0e8e7bf981a0 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:39.069120 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18d9d94-eb94-4afd-86d6-0e8e7bf981a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:39.092787 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-d18d9d94-eb94-4afd-86d6-0e8e7bf981a0 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:39.097698 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-d18d9d94-eb94-4afd-86d6-0e8e7bf981a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:39.105009 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 858/1745] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:39:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:39:39.107550 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d428a3e-f4f3-4c50-933a-5795a5cae8e9 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:39.107550 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d428a3e-f4f3-4c50-933a-5795a5cae8e9 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:39.107550 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d428a3e-f4f3-4c50-933a-5795a5cae8e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:39.112627 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:39:39.130054 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2202793-3a89-40d1-a59e-f32a9de0c191 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:39.134317 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-0d428a3e-f4f3-4c50-933a-5795a5cae8e9 None None] MFA Rules not processed for user `7ac4c85918c045b793a1a53300c989a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:39.136268 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-0d428a3e-f4f3-4c50-933a-5795a5cae8e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:39.136674 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4298d8c6-d501-47cb-b9af-cdd24bc36ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:39.137995 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4298d8c6-d501-47cb-b9af-cdd24bc36ea9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:39.138543 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4298d8c6-d501-47cb-b9af-cdd24bc36ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:39.140698 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4298d8c6-d501-47cb-b9af-cdd24bc36ea9 admin admin] PATH_INFO: `/v3/users/b7c5d5106f674f0d897e42f69012743b` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:39.143608 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4298d8c6-d501-47cb-b9af-cdd24bc36ea9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7c5d5106f674f0d897e42f69012743b)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:39.145667 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4298d8c6-d501-47cb-b9af-cdd24bc36ea9 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:39.220691 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 859/1746] 10.222.0.85 () {58 vars in 1009 bytes} [Tue Jun 7 14:39:39 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:39:39.221551 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d428a3e-f4f3-4c50-933a-5795a5cae8e9 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:39.235194 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d428a3e-f4f3-4c50-933a-5795a5cae8e9 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:39.240812 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e00a60-bf2e-499c-9a2c-9138f6eea949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:39.241496 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e00a60-bf2e-499c-9a2c-9138f6eea949 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:39.241782 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e00a60-bf2e-499c-9a2c-9138f6eea949 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:39.245413 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e00a60-bf2e-499c-9a2c-9138f6eea949 admin admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:39.245413 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e00a60-bf2e-499c-9a2c-9138f6eea949 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:39:39.245413 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e00a60-bf2e-499c-9a2c-9138f6eea949 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:39.245413 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e00a60-bf2e-499c-9a2c-9138f6eea949 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:39.260294 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 860/1747] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jun 7 14:39:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:39:39.260294 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510308d3-11e9-427a-944d-0aac5f2c10cd None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:39.260294 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510308d3-11e9-427a-944d-0aac5f2c10cd None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:39.260294 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510308d3-11e9-427a-944d-0aac5f2c10cd None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:39.261105 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 861/1748] 10.222.0.85 () {54 vars in 921 bytes} [Tue Jun 7 14:39:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:39:39.262608 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510308d3-11e9-427a-944d-0aac5f2c10cd None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:39.264819 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-510308d3-11e9-427a-944d-0aac5f2c10cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:39.278249 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510308d3-11e9-427a-944d-0aac5f2c10cd None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:39.283089 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc980f6-8192-4e31-b302-802a85d1b279 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:39.283953 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc980f6-8192-4e31-b302-802a85d1b279 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:39.284474 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc980f6-8192-4e31-b302-802a85d1b279 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:39.284879 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc980f6-8192-4e31-b302-802a85d1b279 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:39.294130 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc980f6-8192-4e31-b302-802a85d1b279 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:39.295567 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc980f6-8192-4e31-b302-802a85d1b279 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:39.304850 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 862/1749] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:39:39 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:39:39.305662 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc980f6-8192-4e31-b302-802a85d1b279 service neutron] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:39.325496 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc980f6-8192-4e31-b302-802a85d1b279 service neutron] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:39.328957 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1b30ab-3b1e-40f1-a9bc-b939b553053a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:39.329669 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1b30ab-3b1e-40f1-a9bc-b939b553053a admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:39.329937 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1b30ab-3b1e-40f1-a9bc-b939b553053a admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:39.330213 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1b30ab-3b1e-40f1-a9bc-b939b553053a admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:39.330915 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1b30ab-3b1e-40f1-a9bc-b939b553053a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:39.333237 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1b30ab-3b1e-40f1-a9bc-b939b553053a admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:39.365065 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-4298d8c6-d501-47cb-b9af-cdd24bc36ea9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7c5d5106f674f0d897e42f69012743b'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:39.365863 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-4298d8c6-d501-47cb-b9af-cdd24bc36ea9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7c5d5106f674f0d897e42f69012743b'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:39.366649 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-4298d8c6-d501-47cb-b9af-cdd24bc36ea9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7c5d5106f674f0d897e42f69012743b'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:39.382037 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-4298d8c6-d501-47cb-b9af-cdd24bc36ea9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7c5d5106f674f0d897e42f69012743b'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:39.396493 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-4298d8c6-d501-47cb-b9af-cdd24bc36ea9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7c5d5106f674f0d897e42f69012743b'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:39.466383 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 863/1750] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:39:39 2022] POST /identity/v3/projects => generated 355 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:39:39.468765 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee3258d-7aa4-4310-8def-4e7a3ccca218 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:39.469257 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee3258d-7aa4-4310-8def-4e7a3ccca218 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:39.469734 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee3258d-7aa4-4310-8def-4e7a3ccca218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:39.491363 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-4ee3258d-7aa4-4310-8def-4e7a3ccca218 None None] MFA Rules not processed for user `6330294b1c8e4dcd85aad189f007b0aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:39.493613 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-4ee3258d-7aa4-4310-8def-4e7a3ccca218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:39.529201 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 888/1751] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:39:39 2022] DELETE /identity/v3/users/b7c5d5106f674f0d897e42f69012743b => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:39:39.531026 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4298d8c6-d501-47cb-b9af-cdd24bc36ea9 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:39.533642 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-4298d8c6-d501-47cb-b9af-cdd24bc36ea9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:39.583984 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 864/1752] 10.222.0.85 () {58 vars in 1018 bytes} [Tue Jun 7 14:39:39 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:39:39.585478 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef828703-9ac8-4503-8b7c-7346f07400a9 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:39.585834 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef828703-9ac8-4503-8b7c-7346f07400a9 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:39.586181 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef828703-9ac8-4503-8b7c-7346f07400a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:39.608451 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-ef828703-9ac8-4503-8b7c-7346f07400a9 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:39.609790 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-ef828703-9ac8-4503-8b7c-7346f07400a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:39.637626 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4298d8c6-d501-47cb-b9af-cdd24bc36ea9 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:39.644877 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ea434d-7e91-4d60-9556-47e3f66095e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:39.645854 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ea434d-7e91-4d60-9556-47e3f66095e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:39.646570 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ea434d-7e91-4d60-9556-47e3f66095e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:39.646570 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ea434d-7e91-4d60-9556-47e3f66095e1 admin admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:39.647002 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ea434d-7e91-4d60-9556-47e3f66095e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:39.649134 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ea434d-7e91-4d60-9556-47e3f66095e1 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:39.720056 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 865/1753] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:39:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:39:39.721213 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef828703-9ac8-4503-8b7c-7346f07400a9 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:39.723605 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-ef828703-9ac8-4503-8b7c-7346f07400a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:39.791683 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 889/1754] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:39:39 2022] POST /identity/v3/users => generated 278 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:39:39.793438 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ea434d-7e91-4d60-9556-47e3f66095e1 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:39.797615 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-74ea434d-7e91-4d60-9556-47e3f66095e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:39.861989 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef828703-9ac8-4503-8b7c-7346f07400a9 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:39.869659 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa395e8c-9fa2-436e-8981-2d4e7a8e9887 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6330294b1c8e4dcd85aad189f007b0aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:39.874743 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa395e8c-9fa2-436e-8981-2d4e7a8e9887 service placement] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:39.874743 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa395e8c-9fa2-436e-8981-2d4e7a8e9887 service placement] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:39.874743 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa395e8c-9fa2-436e-8981-2d4e7a8e9887 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:39.896926 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa395e8c-9fa2-436e-8981-2d4e7a8e9887 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:39.901602 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa395e8c-9fa2-436e-8981-2d4e7a8e9887 service placement] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:39.914198 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 866/1755] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:39:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:39:39.915246 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa395e8c-9fa2-436e-8981-2d4e7a8e9887 service placement] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:39.928328 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa395e8c-9fa2-436e-8981-2d4e7a8e9887 service placement] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:39.935133 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c237ca-bfcc-41b0-a342-4cbf4498fa42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:39.935873 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c237ca-bfcc-41b0-a342-4cbf4498fa42 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:39.936232 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c237ca-bfcc-41b0-a342-4cbf4498fa42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:39.943709 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c237ca-bfcc-41b0-a342-4cbf4498fa42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:39.946132 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c237ca-bfcc-41b0-a342-4cbf4498fa42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:39.947939 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c237ca-bfcc-41b0-a342-4cbf4498fa42 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:39.984656 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 867/1756] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:39:39 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:39:39.991414 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c237ca-bfcc-41b0-a342-4cbf4498fa42 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:40.033991 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c237ca-bfcc-41b0-a342-4cbf4498fa42 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:40.034663 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ea434d-7e91-4d60-9556-47e3f66095e1 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:40.053312 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6903d494-2ab9-4b79-ad95-56499e1fb6c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:40.054926 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6903d494-2ab9-4b79-ad95-56499e1fb6c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:40.055379 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:39:40.056713 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6903d494-2ab9-4b79-ad95-56499e1fb6c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:40.063315 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6903d494-2ab9-4b79-ad95-56499e1fb6c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:40.063843 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6223310-1f85-4c29-9f43-de8c026dcf69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:40.064149 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6223310-1f85-4c29-9f43-de8c026dcf69 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:40.064473 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6223310-1f85-4c29-9f43-de8c026dcf69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:40.066834 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6223310-1f85-4c29-9f43-de8c026dcf69 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:40.067627 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6223310-1f85-4c29-9f43-de8c026dcf69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:40.071594 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6223310-1f85-4c29-9f43-de8c026dcf69 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:40.094798 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6903d494-2ab9-4b79-ad95-56499e1fb6c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:40.097206 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6903d494-2ab9-4b79-ad95-56499e1fb6c4 service neutron] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:40.126790 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 868/1757] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:39:39 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:39:40.127757 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6223310-1f85-4c29-9f43-de8c026dcf69 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:40.130050 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 890/1758] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:39:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:39:40.150634 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6223310-1f85-4c29-9f43-de8c026dcf69 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:40.158624 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db67c22-87a2-44f2-bb84-ec28436aef95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:40.159579 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db67c22-87a2-44f2-bb84-ec28436aef95 admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:40.159868 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db67c22-87a2-44f2-bb84-ec28436aef95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:40.160145 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db67c22-87a2-44f2-bb84-ec28436aef95 admin admin] PATH_INFO: `/v3/projects/9b7a7c200543413c930cab774ae2c38f/users/80baf17adab84876b85fc67b03ef30f5/roles/3771341b3be641139e752aab690397a4` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:40.188639 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db67c22-87a2-44f2-bb84-ec28436aef95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b7a7c200543413c930cab774ae2c38f, user_id=80baf17adab84876b85fc67b03ef30f5, role_id=3771341b3be641139e752aab690397a4)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:40.194803 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db67c22-87a2-44f2-bb84-ec28436aef95 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:40.281097 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 869/1759] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:39:40 2022] PUT /identity/v3/projects/9b7a7c200543413c930cab774ae2c38f/users/80baf17adab84876b85fc67b03ef30f5/roles/3771341b3be641139e752aab690397a4 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:39:40.292268 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2ebcca-8439-4802-96ad-30aa0fde6936 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:40.292606 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2ebcca-8439-4802-96ad-30aa0fde6936 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:40.292897 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2ebcca-8439-4802-96ad-30aa0fde6936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:40.367286 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-8d2ebcca-8439-4802-96ad-30aa0fde6936 None None] MFA Rules not processed for user `80baf17adab84876b85fc67b03ef30f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:40.369017 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-8d2ebcca-8439-4802-96ad-30aa0fde6936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:40.459638 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 891/1760] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:39:40 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:39:40.464496 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db67c22-87a2-44f2-bb84-ec28436aef95 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:40.544749 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db67c22-87a2-44f2-bb84-ec28436aef95 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:40.550324 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b655ed-4c50-4c0a-a6e0-6d15fb806803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:40.551109 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b655ed-4c50-4c0a-a6e0-6d15fb806803 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:40.551476 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b655ed-4c50-4c0a-a6e0-6d15fb806803 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:40.551829 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b655ed-4c50-4c0a-a6e0-6d15fb806803 admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:40.552982 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b655ed-4c50-4c0a-a6e0-6d15fb806803 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:40.554666 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b655ed-4c50-4c0a-a6e0-6d15fb806803 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:40.691822 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 870/1761] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:39:40 2022] POST /identity/v3/projects => generated 357 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:39:40.696036 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2ebcca-8439-4802-96ad-30aa0fde6936 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:40.714127 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2ebcca-8439-4802-96ad-30aa0fde6936 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:40.789731 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a5ae7-c5fd-4983-8cd2-376625cf6d9f None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:40.790959 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a5ae7-c5fd-4983-8cd2-376625cf6d9f None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:40.791427 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a5ae7-c5fd-4983-8cd2-376625cf6d9f None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:40.793502 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 871/1762] 10.222.0.85 () {54 vars in 921 bytes} [Tue Jun 7 14:39:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:39:40.821313 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b319d19-5f0f-47f5-8284-5a82d18b944b None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:40.821597 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b319d19-5f0f-47f5-8284-5a82d18b944b None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:40.821880 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b319d19-5f0f-47f5-8284-5a82d18b944b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:40.854648 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801aca1c-65e3-4d99-aef4-78e4e6c9e84a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:40.855358 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801aca1c-65e3-4d99-aef4-78e4e6c9e84a admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:40.855778 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801aca1c-65e3-4d99-aef4-78e4e6c9e84a admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:40.856089 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801aca1c-65e3-4d99-aef4-78e4e6c9e84a admin admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:40.856877 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801aca1c-65e3-4d99-aef4-78e4e6c9e84a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:40.861204 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801aca1c-65e3-4d99-aef4-78e4e6c9e84a admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:40.864158 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-2b319d19-5f0f-47f5-8284-5a82d18b944b None None] MFA Rules not processed for user `6330294b1c8e4dcd85aad189f007b0aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:40.866807 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-2b319d19-5f0f-47f5-8284-5a82d18b944b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:40.986026 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 872/1763] 10.222.0.85 () {58 vars in 1018 bytes} [Tue Jun 7 14:39:40 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:39:40.987603 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b319d19-5f0f-47f5-8284-5a82d18b944b None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:40.989145 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-2b319d19-5f0f-47f5-8284-5a82d18b944b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:41.007028 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 892/1764] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:39:40 2022] POST /identity/v3/users => generated 285 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:39:41.009822 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801aca1c-65e3-4d99-aef4-78e4e6c9e84a admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:41.010551 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-801aca1c-65e3-4d99-aef4-78e4e6c9e84a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:41.029763 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801aca1c-65e3-4d99-aef4-78e4e6c9e84a admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:41.037683 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b319d19-5f0f-47f5-8284-5a82d18b944b None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:41.038880 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0951a78-9f92-475f-aedf-d1bbbf1df26a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:41.040587 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0951a78-9f92-475f-aedf-d1bbbf1df26a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:41.042123 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0951a78-9f92-475f-aedf-d1bbbf1df26a admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:41.042610 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0951a78-9f92-475f-aedf-d1bbbf1df26a admin admin] PATH_INFO: `/v3/users/e7d75fce4d694784b5f1c22a491cd79f` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:41.044507 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0951a78-9f92-475f-aedf-d1bbbf1df26a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7d75fce4d694784b5f1c22a491cd79f)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:41.047290 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0951a78-9f92-475f-aedf-d1bbbf1df26a admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:41.062878 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf00bcc-4f7e-4151-9c60-5f37067e8935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:41.066248 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf00bcc-4f7e-4151-9c60-5f37067e8935 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:41.068489 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf00bcc-4f7e-4151-9c60-5f37067e8935 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:41.068860 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf00bcc-4f7e-4151-9c60-5f37067e8935 admin admin] PATH_INFO: `/v3/projects/37770b1a2b4744f3baac63f128fdd77e` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:41.074061 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf00bcc-4f7e-4151-9c60-5f37067e8935 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37770b1a2b4744f3baac63f128fdd77e)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:41.085825 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf00bcc-4f7e-4151-9c60-5f37067e8935 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:41.093586 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:39:41.363789 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:39:41.366193 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-ccf00bcc-4f7e-4151-9c60-5f37067e8935 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37770b1a2b4744f3baac63f128fdd77e', 'request_id': 'req-ccf00bcc-4f7e-4151-9c60-5f37067e8935'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:41.387370 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-e0951a78-9f92-475f-aedf-d1bbbf1df26a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7d75fce4d694784b5f1c22a491cd79f'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:41.388441 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-e0951a78-9f92-475f-aedf-d1bbbf1df26a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7d75fce4d694784b5f1c22a491cd79f'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:41.388756 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-e0951a78-9f92-475f-aedf-d1bbbf1df26a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7d75fce4d694784b5f1c22a491cd79f'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:41.423296 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-e0951a78-9f92-475f-aedf-d1bbbf1df26a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7d75fce4d694784b5f1c22a491cd79f'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:41.454275 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-e0951a78-9f92-475f-aedf-d1bbbf1df26a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7d75fce4d694784b5f1c22a491cd79f'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:41.464082 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-ccf00bcc-4f7e-4151-9c60-5f37067e8935 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37770b1a2b4744f3baac63f128fdd77e', 'request_id': 'req-ccf00bcc-4f7e-4151-9c60-5f37067e8935'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:41.472211 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-ccf00bcc-4f7e-4151-9c60-5f37067e8935 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37770b1a2b4744f3baac63f128fdd77e', 'request_id': 'req-ccf00bcc-4f7e-4151-9c60-5f37067e8935'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:41.518930 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-ccf00bcc-4f7e-4151-9c60-5f37067e8935 admin admin] The token cache is being invalidate because project 37770b1a2b4744f3baac63f128fdd77e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:39:41.519385 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-ccf00bcc-4f7e-4151-9c60-5f37067e8935 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:41.521679 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-ccf00bcc-4f7e-4151-9c60-5f37067e8935 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:41.525780 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 873/1765] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:39:40 2022] DELETE /identity/v3/projects/37770b1a2b4744f3baac63f128fdd77e => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:39:41.528069 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf00bcc-4f7e-4151-9c60-5f37067e8935 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:41.531329 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-ccf00bcc-4f7e-4151-9c60-5f37067e8935 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:41.589464 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 893/1766] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:39:41 2022] DELETE /identity/v3/users/e7d75fce4d694784b5f1c22a491cd79f => generated 0 bytes in 582 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:39:41.591100 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0951a78-9f92-475f-aedf-d1bbbf1df26a admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:41.593471 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-e0951a78-9f92-475f-aedf-d1bbbf1df26a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:41.662180 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf00bcc-4f7e-4151-9c60-5f37067e8935 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:41.736868 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0951a78-9f92-475f-aedf-d1bbbf1df26a admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:41.742543 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe48c45-8748-45af-b871-b44f239c8629 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6330294b1c8e4dcd85aad189f007b0aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:41.743279 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe48c45-8748-45af-b871-b44f239c8629 service placement] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:41.743571 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe48c45-8748-45af-b871-b44f239c8629 service placement] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:41.743853 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe48c45-8748-45af-b871-b44f239c8629 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:41.745920 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-0fe48c45-8748-45af-b871-b44f239c8629 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:41.759110 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe48c45-8748-45af-b871-b44f239c8629 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:41.760950 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe48c45-8748-45af-b871-b44f239c8629 service placement] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:41.776117 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f1b33a-ad48-4361-8ab6-3a2f380856ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:41.776957 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f1b33a-ad48-4361-8ab6-3a2f380856ae admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:41.777420 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f1b33a-ad48-4361-8ab6-3a2f380856ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:41.777951 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f1b33a-ad48-4361-8ab6-3a2f380856ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:41.778790 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f1b33a-ad48-4361-8ab6-3a2f380856ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:41.783065 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f1b33a-ad48-4361-8ab6-3a2f380856ae admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:41.785356 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 894/1767] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:39:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:39:41.788876 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe48c45-8748-45af-b871-b44f239c8629 service placement] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:41.793393 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-0fe48c45-8748-45af-b871-b44f239c8629 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:41.802117 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 874/1768] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:39:41 2022] GET /identity/v3/roles => generated 1420 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:39:41.804122 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f1b33a-ad48-4361-8ab6-3a2f380856ae admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:41.820560 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe48c45-8748-45af-b871-b44f239c8629 service placement] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:41.825682 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f1b33a-ad48-4361-8ab6-3a2f380856ae admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:41.832409 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbd40e6-c4d1-4ead-87cb-a8bc5e499c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:41.833584 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbd40e6-c4d1-4ead-87cb-a8bc5e499c57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:41.833584 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbd40e6-c4d1-4ead-87cb-a8bc5e499c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:41.833797 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbd40e6-c4d1-4ead-87cb-a8bc5e499c57 admin admin] PATH_INFO: `/v3/users/ba5fc1b1ca2b4737b15c486f0b8a63fd` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:41.840419 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbd40e6-c4d1-4ead-87cb-a8bc5e499c57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba5fc1b1ca2b4737b15c486f0b8a63fd)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:41.840619 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e901b57e-4df1-460b-82a3-9261e9650ad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:41.841647 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e901b57e-4df1-460b-82a3-9261e9650ad6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:41.841647 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e901b57e-4df1-460b-82a3-9261e9650ad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:41.841868 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e901b57e-4df1-460b-82a3-9261e9650ad6 admin admin] PATH_INFO: `/v3/projects/9d002e9a174d49b08851bc0eeaf7f81e/users/0695c1cbf77f4162a0feccc9d83495fe/roles/9baff2b18f2649f7b757ff2e1f63c853` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:41.845337 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbd40e6-c4d1-4ead-87cb-a8bc5e499c57 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:41.873768 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e901b57e-4df1-460b-82a3-9261e9650ad6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d002e9a174d49b08851bc0eeaf7f81e, user_id=0695c1cbf77f4162a0feccc9d83495fe, role_id=9baff2b18f2649f7b757ff2e1f63c853)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:41.878007 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e901b57e-4df1-460b-82a3-9261e9650ad6 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:42.011149 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 875/1769] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:39:41 2022] PUT /identity/v3/projects/9d002e9a174d49b08851bc0eeaf7f81e/users/0695c1cbf77f4162a0feccc9d83495fe/roles/9baff2b18f2649f7b757ff2e1f63c853 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:39:42.012321 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e901b57e-4df1-460b-82a3-9261e9650ad6 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:42.015868 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-e901b57e-4df1-460b-82a3-9261e9650ad6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:42.092107 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e901b57e-4df1-460b-82a3-9261e9650ad6 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:42.103553 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41257b94-3c9a-4c70-8d19-c4d7017c35c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:42.108657 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41257b94-3c9a-4c70-8d19-c4d7017c35c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:42.108657 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41257b94-3c9a-4c70-8d19-c4d7017c35c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:42.108657 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41257b94-3c9a-4c70-8d19-c4d7017c35c3 admin admin] PATH_INFO: `/v3/projects/88e4de5957bd439486627366692a8b79` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:42.108657 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41257b94-3c9a-4c70-8d19-c4d7017c35c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88e4de5957bd439486627366692a8b79)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:42.108657 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41257b94-3c9a-4c70-8d19-c4d7017c35c3 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:42.114572 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-bbbd40e6-c4d1-4ead-87cb-a8bc5e499c57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba5fc1b1ca2b4737b15c486f0b8a63fd'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:42.115464 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-bbbd40e6-c4d1-4ead-87cb-a8bc5e499c57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba5fc1b1ca2b4737b15c486f0b8a63fd'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:42.116681 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-bbbd40e6-c4d1-4ead-87cb-a8bc5e499c57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba5fc1b1ca2b4737b15c486f0b8a63fd'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:42.126481 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-bbbd40e6-c4d1-4ead-87cb-a8bc5e499c57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba5fc1b1ca2b4737b15c486f0b8a63fd'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:42.143593 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-bbbd40e6-c4d1-4ead-87cb-a8bc5e499c57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba5fc1b1ca2b4737b15c486f0b8a63fd'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:42.163028 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:39:42.237506 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 895/1770] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:39:41 2022] DELETE /identity/v3/users/ba5fc1b1ca2b4737b15c486f0b8a63fd => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:39:42.238693 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbd40e6-c4d1-4ead-87cb-a8bc5e499c57 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:42.241188 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-bbbd40e6-c4d1-4ead-87cb-a8bc5e499c57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:42.275091 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-41257b94-3c9a-4c70-8d19-c4d7017c35c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88e4de5957bd439486627366692a8b79', 'request_id': 'req-41257b94-3c9a-4c70-8d19-c4d7017c35c3'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:42.340339 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbd40e6-c4d1-4ead-87cb-a8bc5e499c57 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:42.346302 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687a7a8e-0028-4b14-99f7-896cc05b844c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:42.347351 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687a7a8e-0028-4b14-99f7-896cc05b844c admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:42.347868 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687a7a8e-0028-4b14-99f7-896cc05b844c admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:42.348608 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687a7a8e-0028-4b14-99f7-896cc05b844c admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:42.349692 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687a7a8e-0028-4b14-99f7-896cc05b844c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:42.352654 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-41257b94-3c9a-4c70-8d19-c4d7017c35c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88e4de5957bd439486627366692a8b79', 'request_id': 'req-41257b94-3c9a-4c70-8d19-c4d7017c35c3'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:42.356757 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687a7a8e-0028-4b14-99f7-896cc05b844c admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:42.361542 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-41257b94-3c9a-4c70-8d19-c4d7017c35c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88e4de5957bd439486627366692a8b79', 'request_id': 'req-41257b94-3c9a-4c70-8d19-c4d7017c35c3'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:42.374771 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 896/1771] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:39:42 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:39:42.378875 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687a7a8e-0028-4b14-99f7-896cc05b844c admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:42.390966 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-41257b94-3c9a-4c70-8d19-c4d7017c35c3 admin admin] The token cache is being invalidate because project 88e4de5957bd439486627366692a8b79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:39:42.391276 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-41257b94-3c9a-4c70-8d19-c4d7017c35c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:42.392129 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-41257b94-3c9a-4c70-8d19-c4d7017c35c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:42.393150 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687a7a8e-0028-4b14-99f7-896cc05b844c admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:42.399690 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 876/1772] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:39:42 2022] DELETE /identity/v3/projects/88e4de5957bd439486627366692a8b79 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:39:42.401031 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41257b94-3c9a-4c70-8d19-c4d7017c35c3 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:42.402660 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-41257b94-3c9a-4c70-8d19-c4d7017c35c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:42.407906 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdceb196-9242-4f30-bcfe-17d1dd66feae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:42.411081 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdceb196-9242-4f30-bcfe-17d1dd66feae admin admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:42.411389 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdceb196-9242-4f30-bcfe-17d1dd66feae admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:42.412736 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdceb196-9242-4f30-bcfe-17d1dd66feae admin admin] PATH_INFO: `/v3/projects/9d002e9a174d49b08851bc0eeaf7f81e/users/0695c1cbf77f4162a0feccc9d83495fe/roles/9baff2b18f2649f7b757ff2e1f63c853` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:42.415232 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdceb196-9242-4f30-bcfe-17d1dd66feae admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d002e9a174d49b08851bc0eeaf7f81e, user_id=0695c1cbf77f4162a0feccc9d83495fe, role_id=9baff2b18f2649f7b757ff2e1f63c853)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:42.418191 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdceb196-9242-4f30-bcfe-17d1dd66feae admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:42.439892 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 897/1773] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:39:42 2022] PUT /identity/v3/projects/9d002e9a174d49b08851bc0eeaf7f81e/users/0695c1cbf77f4162a0feccc9d83495fe/roles/9baff2b18f2649f7b757ff2e1f63c853 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:39:42.454745 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859e5d03-738f-4397-9ed5-d7c63108ca6f None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:42.455285 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859e5d03-738f-4397-9ed5-d7c63108ca6f None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:42.455682 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859e5d03-738f-4397-9ed5-d7c63108ca6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:42.507745 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41257b94-3c9a-4c70-8d19-c4d7017c35c3 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:42.508782 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-859e5d03-738f-4397-9ed5-d7c63108ca6f None None] MFA Rules not processed for user `0695c1cbf77f4162a0feccc9d83495fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:42.515585 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-859e5d03-738f-4397-9ed5-d7c63108ca6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:42.584187 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebee82f-a6e2-4064-ac53-4ae45d03b061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:42.584354 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebee82f-a6e2-4064-ac53-4ae45d03b061 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:42.584702 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebee82f-a6e2-4064-ac53-4ae45d03b061 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:42.584925 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebee82f-a6e2-4064-ac53-4ae45d03b061 admin admin] PATH_INFO: `/v3/users/50ebe26d884f404eaff5464f199e9a2a` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:42.586345 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebee82f-a6e2-4064-ac53-4ae45d03b061 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50ebe26d884f404eaff5464f199e9a2a)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:42.588293 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebee82f-a6e2-4064-ac53-4ae45d03b061 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:42.742579 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 898/1774] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:39:42 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:39:42.759819 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859e5d03-738f-4397-9ed5-d7c63108ca6f None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:42.771192 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-859e5d03-738f-4397-9ed5-d7c63108ca6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:42.911886 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-7ebee82f-a6e2-4064-ac53-4ae45d03b061 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50ebe26d884f404eaff5464f199e9a2a'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:42.912741 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-7ebee82f-a6e2-4064-ac53-4ae45d03b061 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50ebe26d884f404eaff5464f199e9a2a'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:42.913477 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-7ebee82f-a6e2-4064-ac53-4ae45d03b061 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50ebe26d884f404eaff5464f199e9a2a'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:42.930672 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-7ebee82f-a6e2-4064-ac53-4ae45d03b061 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50ebe26d884f404eaff5464f199e9a2a'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:42.972741 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-7ebee82f-a6e2-4064-ac53-4ae45d03b061 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50ebe26d884f404eaff5464f199e9a2a'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:43.009021 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859e5d03-738f-4397-9ed5-d7c63108ca6f None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:43.020361 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcabe71-7a16-4b2b-a7ba-30bad7e8e515 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d81f597cb44c56a966537089bcce4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:43.024646 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcabe71-7a16-4b2b-a7ba-30bad7e8e515 service cinder] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:43.024646 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcabe71-7a16-4b2b-a7ba-30bad7e8e515 service cinder] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:43.024646 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcabe71-7a16-4b2b-a7ba-30bad7e8e515 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:43.042650 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-8dcabe71-7a16-4b2b-a7ba-30bad7e8e515 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:43.083191 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 877/1775] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:39:42 2022] DELETE /identity/v3/users/50ebe26d884f404eaff5464f199e9a2a => generated 0 bytes in 680 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:39:43.092235 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcabe71-7a16-4b2b-a7ba-30bad7e8e515 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:43.095209 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcabe71-7a16-4b2b-a7ba-30bad7e8e515 service cinder] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:43.147870 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 899/1776] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:39:42 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:39:43.274950 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebee82f-a6e2-4064-ac53-4ae45d03b061 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:43.277539 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-7ebee82f-a6e2-4064-ac53-4ae45d03b061 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:43.466845 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebee82f-a6e2-4064-ac53-4ae45d03b061 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:43.474653 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d472ee8-e3dc-4a51-a4a6-7c07555a1bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:43.477270 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:39:43.478781 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d472ee8-e3dc-4a51-a4a6-7c07555a1bb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:43.479968 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d472ee8-e3dc-4a51-a4a6-7c07555a1bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:43.480596 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d472ee8-e3dc-4a51-a4a6-7c07555a1bb3 admin admin] PATH_INFO: `/v3/projects/4617320ceb90448fa51fe6f46006730b` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:43.486498 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d472ee8-e3dc-4a51-a4a6-7c07555a1bb3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4617320ceb90448fa51fe6f46006730b)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:43.486498 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d472ee8-e3dc-4a51-a4a6-7c07555a1bb3 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:43.544926 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcabe71-7a16-4b2b-a7ba-30bad7e8e515 service cinder] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:43.546444 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-8dcabe71-7a16-4b2b-a7ba-30bad7e8e515 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:43.581635 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcabe71-7a16-4b2b-a7ba-30bad7e8e515 service cinder] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:43.659990 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-0d472ee8-e3dc-4a51-a4a6-7c07555a1bb3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4617320ceb90448fa51fe6f46006730b', 'request_id': 'req-0d472ee8-e3dc-4a51-a4a6-7c07555a1bb3'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:43.713227 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1e81ee-de99-441b-9319-66afe9b47d0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d81f597cb44c56a966537089bcce4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:43.716939 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1e81ee-de99-441b-9319-66afe9b47d0e service cinder] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:43.717363 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1e81ee-de99-441b-9319-66afe9b47d0e service cinder] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:43.717882 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1e81ee-de99-441b-9319-66afe9b47d0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:43.720484 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-7d1e81ee-de99-441b-9319-66afe9b47d0e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:43.825123 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-0d472ee8-e3dc-4a51-a4a6-7c07555a1bb3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4617320ceb90448fa51fe6f46006730b', 'request_id': 'req-0d472ee8-e3dc-4a51-a4a6-7c07555a1bb3'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:43.843351 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-0d472ee8-e3dc-4a51-a4a6-7c07555a1bb3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4617320ceb90448fa51fe6f46006730b', 'request_id': 'req-0d472ee8-e3dc-4a51-a4a6-7c07555a1bb3'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:43.855683 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:39:43.865466 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-0d472ee8-e3dc-4a51-a4a6-7c07555a1bb3 admin admin] The token cache is being invalidate because project 4617320ceb90448fa51fe6f46006730b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:39:43.867087 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-0d472ee8-e3dc-4a51-a4a6-7c07555a1bb3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:43.867983 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-0d472ee8-e3dc-4a51-a4a6-7c07555a1bb3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:43.874994 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 878/1777] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:39:43 2022] DELETE /identity/v3/projects/4617320ceb90448fa51fe6f46006730b => generated 0 bytes in 597 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:39:43.877207 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d472ee8-e3dc-4a51-a4a6-7c07555a1bb3 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:43.882695 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-0d472ee8-e3dc-4a51-a4a6-7c07555a1bb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:43.883091 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1e81ee-de99-441b-9319-66afe9b47d0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:43.892660 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1e81ee-de99-441b-9319-66afe9b47d0e service cinder] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:43.919263 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 900/1778] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:39:43 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 14:39:43.924841 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c9c338-b12a-4e79-9282-85ac79f76a5e None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:43.926358 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c9c338-b12a-4e79-9282-85ac79f76a5e None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:43.926798 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c9c338-b12a-4e79-9282-85ac79f76a5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:43.995303 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-60c9c338-b12a-4e79-9282-85ac79f76a5e None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:44.000770 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-60c9c338-b12a-4e79-9282-85ac79f76a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:44.135935 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d472ee8-e3dc-4a51-a4a6-7c07555a1bb3 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:44.144715 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f526ce8-349f-44be-a2ad-639aa930f543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:44.145970 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f526ce8-349f-44be-a2ad-639aa930f543 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:44.145970 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f526ce8-349f-44be-a2ad-639aa930f543 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:44.146298 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f526ce8-349f-44be-a2ad-639aa930f543 admin admin] PATH_INFO: `/v3/projects/b3ba9817298f4fc9b06bedcbbe8540dc` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:44.148674 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f526ce8-349f-44be-a2ad-639aa930f543 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3ba9817298f4fc9b06bedcbbe8540dc)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:44.151294 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f526ce8-349f-44be-a2ad-639aa930f543 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:44.165809 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 901/1779] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:39:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:39:44.172852 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e34492-adb1-4a56-a861-fbbe1e331fe6 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:44.173377 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e34492-adb1-4a56-a861-fbbe1e331fe6 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:44.174013 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e34492-adb1-4a56-a861-fbbe1e331fe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:44.212114 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-62e34492-adb1-4a56-a861-fbbe1e331fe6 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:44.214632 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-62e34492-adb1-4a56-a861-fbbe1e331fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:44.230062 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 902/1780] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:39:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:39:44.235445 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e34492-adb1-4a56-a861-fbbe1e331fe6 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:44.260948 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e34492-adb1-4a56-a861-fbbe1e331fe6 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:44.271571 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aca1d99-e4b5-41c2-9458-7b4ebea5a86c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:44.272319 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aca1d99-e4b5-41c2-9458-7b4ebea5a86c admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:44.273035 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aca1d99-e4b5-41c2-9458-7b4ebea5a86c admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:44.273322 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aca1d99-e4b5-41c2-9458-7b4ebea5a86c admin admin] PATH_INFO: `/v3/domains` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:44.274242 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aca1d99-e4b5-41c2-9458-7b4ebea5a86c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:39:44.274560 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aca1d99-e4b5-41c2-9458-7b4ebea5a86c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:44.276940 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aca1d99-e4b5-41c2-9458-7b4ebea5a86c admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:44.316467 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 903/1781] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jun 7 14:39:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:39:44.323596 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aca1d99-e4b5-41c2-9458-7b4ebea5a86c admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:44.365810 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-5f526ce8-349f-44be-a2ad-639aa930f543 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3ba9817298f4fc9b06bedcbbe8540dc', 'request_id': 'req-5f526ce8-349f-44be-a2ad-639aa930f543'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:44.367007 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aca1d99-e4b5-41c2-9458-7b4ebea5a86c admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:44.461236 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40723d68-c8f6-499a-9d42-73929eaa6abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:44.462080 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40723d68-c8f6-499a-9d42-73929eaa6abd admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:44.462508 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40723d68-c8f6-499a-9d42-73929eaa6abd admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:44.462875 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40723d68-c8f6-499a-9d42-73929eaa6abd admin admin] PATH_INFO: `/v3/projects` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:44.463714 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40723d68-c8f6-499a-9d42-73929eaa6abd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:44.465887 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40723d68-c8f6-499a-9d42-73929eaa6abd admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:44.471277 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-5f526ce8-349f-44be-a2ad-639aa930f543 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3ba9817298f4fc9b06bedcbbe8540dc', 'request_id': 'req-5f526ce8-349f-44be-a2ad-639aa930f543'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:44.481558 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-5f526ce8-349f-44be-a2ad-639aa930f543 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3ba9817298f4fc9b06bedcbbe8540dc', 'request_id': 'req-5f526ce8-349f-44be-a2ad-639aa930f543'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:44.502567 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-5f526ce8-349f-44be-a2ad-639aa930f543 admin admin] The token cache is being invalidate because project b3ba9817298f4fc9b06bedcbbe8540dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:39:44.503211 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-5f526ce8-349f-44be-a2ad-639aa930f543 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:44.504193 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-5f526ce8-349f-44be-a2ad-639aa930f543 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:44.510290 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 879/1782] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:39:43 2022] DELETE /identity/v3/projects/b3ba9817298f4fc9b06bedcbbe8540dc => generated 0 bytes in 631 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:39:44.512364 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a06414-33cf-4fc2-9014-87be4af35c17 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:44.512729 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a06414-33cf-4fc2-9014-87be4af35c17 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:44.516946 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a06414-33cf-4fc2-9014-87be4af35c17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:44.533462 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-22a06414-33cf-4fc2-9014-87be4af35c17 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:44.534977 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-22a06414-33cf-4fc2-9014-87be4af35c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:44.551574 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 880/1783] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:39:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:39:44.559290 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239d421c-8247-4164-a8bd-cbe8486be790 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:44.559635 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239d421c-8247-4164-a8bd-cbe8486be790 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:44.559937 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239d421c-8247-4164-a8bd-cbe8486be790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:44.581754 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 904/1784] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:39:44 2022] POST /identity/v3/projects => generated 387 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:39:44.583904 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40723d68-c8f6-499a-9d42-73929eaa6abd admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:44.586209 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-40723d68-c8f6-499a-9d42-73929eaa6abd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:44.620480 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-239d421c-8247-4164-a8bd-cbe8486be790 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:44.621514 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-239d421c-8247-4164-a8bd-cbe8486be790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:44.744915 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 881/1785] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:39:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:39:44.747840 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239d421c-8247-4164-a8bd-cbe8486be790 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:44.757568 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40723d68-c8f6-499a-9d42-73929eaa6abd admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:44.767906 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8adff7-4193-4eea-93d9-7858f09b9390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:44.771067 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8adff7-4193-4eea-93d9-7858f09b9390 admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:44.771067 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8adff7-4193-4eea-93d9-7858f09b9390 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:44.771067 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8adff7-4193-4eea-93d9-7858f09b9390 admin admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:44.771067 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8adff7-4193-4eea-93d9-7858f09b9390 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:44.772485 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8adff7-4193-4eea-93d9-7858f09b9390 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:44.781066 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239d421c-8247-4164-a8bd-cbe8486be790 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:44.786855 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3788f5c1-31a5-4063-8baf-20504504e337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:44.787533 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3788f5c1-31a5-4063-8baf-20504504e337 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:44.787831 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3788f5c1-31a5-4063-8baf-20504504e337 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:44.788284 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3788f5c1-31a5-4063-8baf-20504504e337 admin admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:44.789066 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3788f5c1-31a5-4063-8baf-20504504e337 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:39:44.789383 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3788f5c1-31a5-4063-8baf-20504504e337 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:44.791140 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3788f5c1-31a5-4063-8baf-20504504e337 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:44.819589 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 882/1786] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jun 7 14:39:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:39:44.826391 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3788f5c1-31a5-4063-8baf-20504504e337 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:44.837296 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3788f5c1-31a5-4063-8baf-20504504e337 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:44.850158 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91122f6-85bd-455b-9e77-687a078951bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:44.850861 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91122f6-85bd-455b-9e77-687a078951bb admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:44.851145 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91122f6-85bd-455b-9e77-687a078951bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:44.851426 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91122f6-85bd-455b-9e77-687a078951bb admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:44.852205 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91122f6-85bd-455b-9e77-687a078951bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:44.854118 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91122f6-85bd-455b-9e77-687a078951bb admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:44.967432 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 905/1787] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:39:44 2022] POST /identity/v3/users => generated 294 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:39:44.968642 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8adff7-4193-4eea-93d9-7858f09b9390 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:44.979048 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8adff7-4193-4eea-93d9-7858f09b9390 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:44.983887 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e025b24-90a3-40a4-9d6d-29746c869177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:44.984589 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e025b24-90a3-40a4-9d6d-29746c869177 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:44.984874 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e025b24-90a3-40a4-9d6d-29746c869177 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:44.985148 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e025b24-90a3-40a4-9d6d-29746c869177 admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:44.985973 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e025b24-90a3-40a4-9d6d-29746c869177 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:44.987471 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e025b24-90a3-40a4-9d6d-29746c869177 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:44.999592 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 906/1788] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:39:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 14:39:45.013129 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e025b24-90a3-40a4-9d6d-29746c869177 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:45.029040 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e025b24-90a3-40a4-9d6d-29746c869177 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:45.035445 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87775ba6-58a9-4d60-b1f5-4257f28c424a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:45.042602 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87775ba6-58a9-4d60-b1f5-4257f28c424a admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:45.042904 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87775ba6-58a9-4d60-b1f5-4257f28c424a admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:45.043202 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87775ba6-58a9-4d60-b1f5-4257f28c424a admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:45.043986 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87775ba6-58a9-4d60-b1f5-4257f28c424a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:45.045692 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87775ba6-58a9-4d60-b1f5-4257f28c424a admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:45.065491 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 907/1789] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:39:45 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 14:39:45.066578 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87775ba6-58a9-4d60-b1f5-4257f28c424a admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:45.078475 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87775ba6-58a9-4d60-b1f5-4257f28c424a admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:45.083894 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598d3d8c-f208-4b2f-89a1-25b1658d629a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:45.084712 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598d3d8c-f208-4b2f-89a1-25b1658d629a admin admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:45.084937 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598d3d8c-f208-4b2f-89a1-25b1658d629a admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:45.085256 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598d3d8c-f208-4b2f-89a1-25b1658d629a admin admin] PATH_INFO: `/v3/projects/e605046e2c214fdf9fc0c821acda2d4e/users/5a2986dc97c14e1bb81fe2c6971baec4/roles/3771341b3be641139e752aab690397a4` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:45.113253 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598d3d8c-f208-4b2f-89a1-25b1658d629a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e605046e2c214fdf9fc0c821acda2d4e, user_id=5a2986dc97c14e1bb81fe2c6971baec4, role_id=3771341b3be641139e752aab690397a4)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:45.117539 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598d3d8c-f208-4b2f-89a1-25b1658d629a admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:45.183017 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 883/1790] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:39:44 2022] POST /identity/v3/projects => generated 371 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:39:45.187756 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91122f6-85bd-455b-9e77-687a078951bb admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:45.300027 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91122f6-85bd-455b-9e77-687a078951bb admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:45.308561 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c57633-1d2e-4dd8-af50-471adede8dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:45.309270 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c57633-1d2e-4dd8-af50-471adede8dc1 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:45.309559 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c57633-1d2e-4dd8-af50-471adede8dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:45.309842 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c57633-1d2e-4dd8-af50-471adede8dc1 admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:45.310549 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c57633-1d2e-4dd8-af50-471adede8dc1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:45.347990 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c57633-1d2e-4dd8-af50-471adede8dc1 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:45.593396 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 884/1791] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:39:45 2022] POST /identity/v3/users => generated 286 bytes in 407 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:39:45.598771 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c57633-1d2e-4dd8-af50-471adede8dc1 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:45.611891 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c57633-1d2e-4dd8-af50-471adede8dc1 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:45.626550 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 908/1792] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:39:45 2022] PUT /identity/v3/projects/e605046e2c214fdf9fc0c821acda2d4e/users/5a2986dc97c14e1bb81fe2c6971baec4/roles/3771341b3be641139e752aab690397a4 => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:39:45.632332 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b646c7-a6bb-417c-b929-e110ae107978 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:45.632696 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b646c7-a6bb-417c-b929-e110ae107978 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:45.633361 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b646c7-a6bb-417c-b929-e110ae107978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:45.718483 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-01b646c7-a6bb-417c-b929-e110ae107978 None None] MFA Rules not processed for user `5a2986dc97c14e1bb81fe2c6971baec4`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:45.718483 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-01b646c7-a6bb-417c-b929-e110ae107978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:45.727665 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0e79df-892a-4014-b3db-77dd2988423e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:45.728634 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0e79df-892a-4014-b3db-77dd2988423e admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:45.728734 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0e79df-892a-4014-b3db-77dd2988423e admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:45.730044 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0e79df-892a-4014-b3db-77dd2988423e admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:45.730803 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0e79df-892a-4014-b3db-77dd2988423e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:45.732455 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0e79df-892a-4014-b3db-77dd2988423e admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:45.757741 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 885/1793] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:39:45 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 14:39:45.759163 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0e79df-892a-4014-b3db-77dd2988423e admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:45.778365 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0e79df-892a-4014-b3db-77dd2988423e admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:45.781588 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e43a20c-6885-4e03-b300-b9e2eb371fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:45.782294 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e43a20c-6885-4e03-b300-b9e2eb371fc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:45.782648 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e43a20c-6885-4e03-b300-b9e2eb371fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:45.782939 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e43a20c-6885-4e03-b300-b9e2eb371fc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:45.783732 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e43a20c-6885-4e03-b300-b9e2eb371fc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:45.785517 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e43a20c-6885-4e03-b300-b9e2eb371fc3 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:45.797152 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 886/1794] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:39:45 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:39:45.807015 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e43a20c-6885-4e03-b300-b9e2eb371fc3 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:45.815442 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e43a20c-6885-4e03-b300-b9e2eb371fc3 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:45.823698 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed99d56f-06cc-47e3-8dd3-99dced5d390b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:45.824934 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed99d56f-06cc-47e3-8dd3-99dced5d390b admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:45.825235 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed99d56f-06cc-47e3-8dd3-99dced5d390b admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:45.825517 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed99d56f-06cc-47e3-8dd3-99dced5d390b admin admin] PATH_INFO: `/v3/projects/05644c2217174db280f9b50d8b4628bc/users/e3a06e1b6985429d98dcf9aa0c88e741/roles/3771341b3be641139e752aab690397a4` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:45.835411 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 909/1795] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:39:45 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:39:45.849138 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed99d56f-06cc-47e3-8dd3-99dced5d390b admin admin] RBAC: Authorizing `identity:create_grant(project_id=05644c2217174db280f9b50d8b4628bc, user_id=e3a06e1b6985429d98dcf9aa0c88e741, role_id=3771341b3be641139e752aab690397a4)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:45.849540 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b646c7-a6bb-417c-b929-e110ae107978 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:45.853795 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed99d56f-06cc-47e3-8dd3-99dced5d390b admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:45.863692 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b646c7-a6bb-417c-b929-e110ae107978 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:45.869094 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8af8981-bf28-41d1-898a-274bcde4c173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:45.869888 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8af8981-bf28-41d1-898a-274bcde4c173 admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:45.870247 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8af8981-bf28-41d1-898a-274bcde4c173 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:45.870625 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8af8981-bf28-41d1-898a-274bcde4c173 admin admin] PATH_INFO: `/v3/projects` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:45.871415 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8af8981-bf28-41d1-898a-274bcde4c173 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:45.875076 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8af8981-bf28-41d1-898a-274bcde4c173 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:45.980788 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 887/1796] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:39:45 2022] PUT /identity/v3/projects/05644c2217174db280f9b50d8b4628bc/users/e3a06e1b6985429d98dcf9aa0c88e741/roles/3771341b3be641139e752aab690397a4 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:39:45.984658 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 910/1797] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:39:45 2022] POST /identity/v3/projects => generated 385 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:39:45.987069 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed99d56f-06cc-47e3-8dd3-99dced5d390b admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:45.994047 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb71385-ed70-4fef-b8aa-4f17d000f55c None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:45.994333 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb71385-ed70-4fef-b8aa-4f17d000f55c None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:45.994612 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb71385-ed70-4fef-b8aa-4f17d000f55c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:46.021416 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-ebb71385-ed70-4fef-b8aa-4f17d000f55c None None] MFA Rules not processed for user `e3a06e1b6985429d98dcf9aa0c88e741`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:46.022620 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-ebb71385-ed70-4fef-b8aa-4f17d000f55c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:46.063907 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed99d56f-06cc-47e3-8dd3-99dced5d390b admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:46.089859 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 911/1798] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:39:45 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:39:46.095072 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb71385-ed70-4fef-b8aa-4f17d000f55c None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:46.123860 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16c84c5-97de-4308-b7cd-8bfc0cdb4795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:46.124596 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16c84c5-97de-4308-b7cd-8bfc0cdb4795 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:46.124886 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16c84c5-97de-4308-b7cd-8bfc0cdb4795 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:46.125170 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16c84c5-97de-4308-b7cd-8bfc0cdb4795 admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:46.125938 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16c84c5-97de-4308-b7cd-8bfc0cdb4795 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:46.127824 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16c84c5-97de-4308-b7cd-8bfc0cdb4795 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:46.157870 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb71385-ed70-4fef-b8aa-4f17d000f55c None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:46.163230 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc7f8b1-e3f2-47f1-93dd-1fe34d6846a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:46.163887 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc7f8b1-e3f2-47f1-93dd-1fe34d6846a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:46.164182 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc7f8b1-e3f2-47f1-93dd-1fe34d6846a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:46.164486 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc7f8b1-e3f2-47f1-93dd-1fe34d6846a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:46.165179 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc7f8b1-e3f2-47f1-93dd-1fe34d6846a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:46.167187 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc7f8b1-e3f2-47f1-93dd-1fe34d6846a1 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:46.220485 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 888/1799] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:39:45 2022] POST /identity/v3/users => generated 293 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:39:46.223742 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16c84c5-97de-4308-b7cd-8bfc0cdb4795 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:46.236731 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16c84c5-97de-4308-b7cd-8bfc0cdb4795 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:46.246868 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8320b56b-bf8b-43a3-96f8-22cd45cfe6ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:46.253641 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8320b56b-bf8b-43a3-96f8-22cd45cfe6ec admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:46.253641 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8320b56b-bf8b-43a3-96f8-22cd45cfe6ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:46.253641 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8320b56b-bf8b-43a3-96f8-22cd45cfe6ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:46.253641 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8320b56b-bf8b-43a3-96f8-22cd45cfe6ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:46.253641 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8320b56b-bf8b-43a3-96f8-22cd45cfe6ec admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:46.261301 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 889/1800] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:39:46 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 14:39:46.265868 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8320b56b-bf8b-43a3-96f8-22cd45cfe6ec admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:46.276062 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8320b56b-bf8b-43a3-96f8-22cd45cfe6ec admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:46.284220 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 912/1801] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:39:46 2022] POST /identity/v3/projects => generated 371 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:39:46.284335 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348d623d-121f-4380-abc3-a9df40551f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:46.284335 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348d623d-121f-4380-abc3-a9df40551f1c admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:46.284335 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348d623d-121f-4380-abc3-a9df40551f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:46.284335 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348d623d-121f-4380-abc3-a9df40551f1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:46.284684 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348d623d-121f-4380-abc3-a9df40551f1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:46.284839 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348d623d-121f-4380-abc3-a9df40551f1c admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:46.285241 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc7f8b1-e3f2-47f1-93dd-1fe34d6846a1 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:46.295796 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 890/1802] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:39:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 14:39:46.301310 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348d623d-121f-4380-abc3-a9df40551f1c admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:46.310929 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348d623d-121f-4380-abc3-a9df40551f1c admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:46.351799 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc7f8b1-e3f2-47f1-93dd-1fe34d6846a1 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:46.356557 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68acad3b-8123-40c3-8eb8-e41ee468c1ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:46.357222 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68acad3b-8123-40c3-8eb8-e41ee468c1ac admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:46.357506 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68acad3b-8123-40c3-8eb8-e41ee468c1ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:46.357782 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68acad3b-8123-40c3-8eb8-e41ee468c1ac admin admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:46.358845 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68acad3b-8123-40c3-8eb8-e41ee468c1ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:46.360804 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68acad3b-8123-40c3-8eb8-e41ee468c1ac admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:46.372669 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f555974d-cfe6-41ed-bd71-408a34c31668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:46.373707 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f555974d-cfe6-41ed-bd71-408a34c31668 admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:46.374399 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f555974d-cfe6-41ed-bd71-408a34c31668 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:46.374399 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f555974d-cfe6-41ed-bd71-408a34c31668 admin admin] PATH_INFO: `/v3/projects/9218efbcd2824c769e2101488c9f3c57/users/6389fa395ba74b6cb646df091cb1c544/roles/3771341b3be641139e752aab690397a4` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:46.391405 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f555974d-cfe6-41ed-bd71-408a34c31668 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9218efbcd2824c769e2101488c9f3c57, user_id=6389fa395ba74b6cb646df091cb1c544, role_id=3771341b3be641139e752aab690397a4)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:46.394083 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f555974d-cfe6-41ed-bd71-408a34c31668 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:46.443023 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 913/1803] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:39:46 2022] POST /identity/v3/users => generated 292 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:39:46.446970 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68acad3b-8123-40c3-8eb8-e41ee468c1ac admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:46.457889 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68acad3b-8123-40c3-8eb8-e41ee468c1ac admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:46.464137 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:39:46.469257 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5c1d4a-6e31-4358-a733-8b783538e4c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:46.469634 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5c1d4a-6e31-4358-a733-8b783538e4c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:46.469874 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5c1d4a-6e31-4358-a733-8b783538e4c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:46.470159 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5c1d4a-6e31-4358-a733-8b783538e4c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:46.471069 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5c1d4a-6e31-4358-a733-8b783538e4c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:46.473852 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 891/1804] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:39:46 2022] PUT /identity/v3/projects/9218efbcd2824c769e2101488c9f3c57/users/6389fa395ba74b6cb646df091cb1c544/roles/3771341b3be641139e752aab690397a4 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:39:46.475537 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5c1d4a-6e31-4358-a733-8b783538e4c8 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:46.485957 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4238b97-7602-40e9-9d69-e634398e7881 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:46.486428 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4238b97-7602-40e9-9d69-e634398e7881 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:46.486798 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4238b97-7602-40e9-9d69-e634398e7881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:46.492918 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 914/1805] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:39:46 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 14:39:46.500181 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5c1d4a-6e31-4358-a733-8b783538e4c8 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:46.510422 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5c1d4a-6e31-4358-a733-8b783538e4c8 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:46.522109 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-d4238b97-7602-40e9-9d69-e634398e7881 None None] MFA Rules not processed for user `6389fa395ba74b6cb646df091cb1c544`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:46.526164 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-d4238b97-7602-40e9-9d69-e634398e7881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:46.572518 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18a0f05-455f-4194-b535-87f86728052d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:46.573567 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18a0f05-455f-4194-b535-87f86728052d admin admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:46.573998 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18a0f05-455f-4194-b535-87f86728052d admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:46.574284 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18a0f05-455f-4194-b535-87f86728052d admin admin] PATH_INFO: `/v3/projects/a793d314b8a544bb9ad12d7474f0e22a/users/65b0fd324e044e018311ccb2eed9262f/roles/9baff2b18f2649f7b757ff2e1f63c853` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:46.589459 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18a0f05-455f-4194-b535-87f86728052d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a793d314b8a544bb9ad12d7474f0e22a, user_id=65b0fd324e044e018311ccb2eed9262f, role_id=9baff2b18f2649f7b757ff2e1f63c853)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:46.596506 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18a0f05-455f-4194-b535-87f86728052d admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:46.597181 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 892/1806] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:39:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:39:46.601498 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4238b97-7602-40e9-9d69-e634398e7881 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:46.611864 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4238b97-7602-40e9-9d69-e634398e7881 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:46.616300 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f676a35-fcd1-4321-a771-afb9b774032e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:46.616968 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f676a35-fcd1-4321-a771-afb9b774032e admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:46.617253 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f676a35-fcd1-4321-a771-afb9b774032e admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:46.617527 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f676a35-fcd1-4321-a771-afb9b774032e admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:46.618203 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f676a35-fcd1-4321-a771-afb9b774032e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:46.620148 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f676a35-fcd1-4321-a771-afb9b774032e admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:46.719151 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:39:46.727705 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 915/1807] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:39:46 2022] PUT /identity/v3/projects/a793d314b8a544bb9ad12d7474f0e22a/users/65b0fd324e044e018311ccb2eed9262f/roles/9baff2b18f2649f7b757ff2e1f63c853 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:39:46.733104 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18a0f05-455f-4194-b535-87f86728052d admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:46.733287 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 893/1808] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:39:46 2022] POST /identity/v3/projects => generated 387 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:39:46.736334 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f676a35-fcd1-4321-a771-afb9b774032e admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:46.816003 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18a0f05-455f-4194-b535-87f86728052d admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:46.823312 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d572158c-4bae-4725-a652-fbadb9d8b41e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:46.824003 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f676a35-fcd1-4321-a771-afb9b774032e admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:46.824295 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d572158c-4bae-4725-a652-fbadb9d8b41e admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:46.824833 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d572158c-4bae-4725-a652-fbadb9d8b41e admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:46.825200 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d572158c-4bae-4725-a652-fbadb9d8b41e admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:46.826606 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d572158c-4bae-4725-a652-fbadb9d8b41e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:46.830028 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d572158c-4bae-4725-a652-fbadb9d8b41e admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:46.830160 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4135aea6-4640-4ebb-abc1-b027192783a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:46.830845 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4135aea6-4640-4ebb-abc1-b027192783a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:46.831136 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4135aea6-4640-4ebb-abc1-b027192783a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:46.831424 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4135aea6-4640-4ebb-abc1-b027192783a0 admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:46.832340 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4135aea6-4640-4ebb-abc1-b027192783a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:46.834363 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4135aea6-4640-4ebb-abc1-b027192783a0 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:46.842081 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 916/1809] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:39:46 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:39:46.857694 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d572158c-4bae-4725-a652-fbadb9d8b41e admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:46.869007 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d572158c-4bae-4725-a652-fbadb9d8b41e admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:46.877865 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b369a7dc-0730-47f4-9665-9cdba27c4b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:46.877865 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b369a7dc-0730-47f4-9665-9cdba27c4b64 admin admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:46.877865 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b369a7dc-0730-47f4-9665-9cdba27c4b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:46.877865 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b369a7dc-0730-47f4-9665-9cdba27c4b64 admin admin] PATH_INFO: `/v3/projects/a793d314b8a544bb9ad12d7474f0e22a/users/65b0fd324e044e018311ccb2eed9262f/roles/9baff2b18f2649f7b757ff2e1f63c853` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:46.879657 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b369a7dc-0730-47f4-9665-9cdba27c4b64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a793d314b8a544bb9ad12d7474f0e22a, user_id=65b0fd324e044e018311ccb2eed9262f, role_id=9baff2b18f2649f7b757ff2e1f63c853)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:46.883198 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b369a7dc-0730-47f4-9665-9cdba27c4b64 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:46.906471 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 917/1810] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:39:46 2022] PUT /identity/v3/projects/a793d314b8a544bb9ad12d7474f0e22a/users/65b0fd324e044e018311ccb2eed9262f/roles/9baff2b18f2649f7b757ff2e1f63c853 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:39:46.916837 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab54fc8c-7edf-48f1-ba1c-5abc3489ebea None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:46.916837 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab54fc8c-7edf-48f1-ba1c-5abc3489ebea None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:46.916837 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab54fc8c-7edf-48f1-ba1c-5abc3489ebea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:46.949678 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-ab54fc8c-7edf-48f1-ba1c-5abc3489ebea None None] MFA Rules not processed for user `65b0fd324e044e018311ccb2eed9262f`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:46.949678 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-ab54fc8c-7edf-48f1-ba1c-5abc3489ebea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:46.981813 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 894/1811] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:39:46 2022] POST /identity/v3/users => generated 300 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:39:46.985661 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4135aea6-4640-4ebb-abc1-b027192783a0 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:46.998589 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4135aea6-4640-4ebb-abc1-b027192783a0 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:47.063272 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 918/1812] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:39:46 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:39:47.075194 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab54fc8c-7edf-48f1-ba1c-5abc3489ebea None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:47.076802 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-ab54fc8c-7edf-48f1-ba1c-5abc3489ebea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:47.083998 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a7131a-8554-481b-a2ff-9b7bb8d0a753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:47.084724 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a7131a-8554-481b-a2ff-9b7bb8d0a753 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:47.085013 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a7131a-8554-481b-a2ff-9b7bb8d0a753 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:47.085294 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a7131a-8554-481b-a2ff-9b7bb8d0a753 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:47.086038 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a7131a-8554-481b-a2ff-9b7bb8d0a753 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:47.087643 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a7131a-8554-481b-a2ff-9b7bb8d0a753 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:47.099184 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 895/1813] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:39:46 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:39:47.106034 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a7131a-8554-481b-a2ff-9b7bb8d0a753 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:47.118037 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a7131a-8554-481b-a2ff-9b7bb8d0a753 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:47.123699 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04ee5f7-875f-4d31-99d9-546cafca3a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:47.125203 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04ee5f7-875f-4d31-99d9-546cafca3a39 admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:47.125492 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04ee5f7-875f-4d31-99d9-546cafca3a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:47.125772 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04ee5f7-875f-4d31-99d9-546cafca3a39 admin admin] PATH_INFO: `/v3/projects/9c68a04094a945f59ec7fbbd43ffe086/users/330bfaa1f3924f6fa0c4f27c28e1834e/roles/9baff2b18f2649f7b757ff2e1f63c853` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:47.141890 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab54fc8c-7edf-48f1-ba1c-5abc3489ebea None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:47.142236 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04ee5f7-875f-4d31-99d9-546cafca3a39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c68a04094a945f59ec7fbbd43ffe086, user_id=330bfaa1f3924f6fa0c4f27c28e1834e, role_id=9baff2b18f2649f7b757ff2e1f63c853)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:47.145054 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04ee5f7-875f-4d31-99d9-546cafca3a39 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:47.145645 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b563ddd0-a745-4310-875b-f7d477aace1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d81f597cb44c56a966537089bcce4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:47.146932 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b563ddd0-a745-4310-875b-f7d477aace1e service cinder] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:47.147223 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b563ddd0-a745-4310-875b-f7d477aace1e service cinder] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:47.147503 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b563ddd0-a745-4310-875b-f7d477aace1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:47.156814 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b563ddd0-a745-4310-875b-f7d477aace1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:47.158286 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b563ddd0-a745-4310-875b-f7d477aace1e service cinder] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:47.167988 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 919/1814] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:39:47 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:39:47.168808 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b563ddd0-a745-4310-875b-f7d477aace1e service cinder] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:47.170086 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-b563ddd0-a745-4310-875b-f7d477aace1e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:47.185984 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b563ddd0-a745-4310-875b-f7d477aace1e service cinder] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:47.193108 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec861a8d-c000-492b-b977-df71942ed461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:47.194603 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec861a8d-c000-492b-b977-df71942ed461 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:47.194817 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec861a8d-c000-492b-b977-df71942ed461 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:47.195235 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec861a8d-c000-492b-b977-df71942ed461 admin admin] PATH_INFO: `/v3/users/4b110006de2d434789d1d7fda2d73c6c` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:47.197046 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec861a8d-c000-492b-b977-df71942ed461 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b110006de2d434789d1d7fda2d73c6c)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:47.200042 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec861a8d-c000-492b-b977-df71942ed461 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:47.236952 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 896/1815] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:39:47 2022] PUT /identity/v3/projects/9c68a04094a945f59ec7fbbd43ffe086/users/330bfaa1f3924f6fa0c4f27c28e1834e/roles/9baff2b18f2649f7b757ff2e1f63c853 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:39:47.240926 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04ee5f7-875f-4d31-99d9-546cafca3a39 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:47.340405 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04ee5f7-875f-4d31-99d9-546cafca3a39 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:47.345955 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f688210-98cf-48f1-ad6c-bd9ff81b8f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:47.346827 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f688210-98cf-48f1-ad6c-bd9ff81b8f76 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:47.347221 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f688210-98cf-48f1-ad6c-bd9ff81b8f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:47.347604 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f688210-98cf-48f1-ad6c-bd9ff81b8f76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:47.348657 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f688210-98cf-48f1-ad6c-bd9ff81b8f76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:47.350741 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f688210-98cf-48f1-ad6c-bd9ff81b8f76 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:47.366886 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 897/1816] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:39:47 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 14:39:47.370347 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f688210-98cf-48f1-ad6c-bd9ff81b8f76 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:47.382689 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f688210-98cf-48f1-ad6c-bd9ff81b8f76 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:47.388886 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971c4231-e14d-408d-aef9-015ac8a190b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:47.392055 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971c4231-e14d-408d-aef9-015ac8a190b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:47.392556 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971c4231-e14d-408d-aef9-015ac8a190b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:47.392952 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971c4231-e14d-408d-aef9-015ac8a190b8 admin admin] PATH_INFO: `/v3/projects/9c68a04094a945f59ec7fbbd43ffe086/users/330bfaa1f3924f6fa0c4f27c28e1834e/roles/9baff2b18f2649f7b757ff2e1f63c853` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:47.396951 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971c4231-e14d-408d-aef9-015ac8a190b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c68a04094a945f59ec7fbbd43ffe086, user_id=330bfaa1f3924f6fa0c4f27c28e1834e, role_id=9baff2b18f2649f7b757ff2e1f63c853)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:47.399492 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971c4231-e14d-408d-aef9-015ac8a190b8 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:47.416017 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 898/1817] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:39:47 2022] PUT /identity/v3/projects/9c68a04094a945f59ec7fbbd43ffe086/users/330bfaa1f3924f6fa0c4f27c28e1834e/roles/9baff2b18f2649f7b757ff2e1f63c853 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:39:47.432252 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34741cf-a371-4aa0-8cb4-8076658e23d3 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:47.433065 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34741cf-a371-4aa0-8cb4-8076658e23d3 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:47.433546 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34741cf-a371-4aa0-8cb4-8076658e23d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:47.470654 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-a34741cf-a371-4aa0-8cb4-8076658e23d3 None None] MFA Rules not processed for user `330bfaa1f3924f6fa0c4f27c28e1834e`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:47.472493 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-a34741cf-a371-4aa0-8cb4-8076658e23d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:47.477583 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-ec861a8d-c000-492b-b977-df71942ed461 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b110006de2d434789d1d7fda2d73c6c'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:47.478482 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-ec861a8d-c000-492b-b977-df71942ed461 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b110006de2d434789d1d7fda2d73c6c'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:47.479489 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-ec861a8d-c000-492b-b977-df71942ed461 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b110006de2d434789d1d7fda2d73c6c'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:47.495818 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-ec861a8d-c000-492b-b977-df71942ed461 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b110006de2d434789d1d7fda2d73c6c'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:47.527066 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-ec861a8d-c000-492b-b977-df71942ed461 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b110006de2d434789d1d7fda2d73c6c'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:47.596828 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 899/1818] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:39:47 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:39:47.597744 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34741cf-a371-4aa0-8cb4-8076658e23d3 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:47.599383 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-a34741cf-a371-4aa0-8cb4-8076658e23d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:47.678882 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 920/1819] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:39:47 2022] DELETE /identity/v3/users/4b110006de2d434789d1d7fda2d73c6c => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:39:47.680854 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34741cf-a371-4aa0-8cb4-8076658e23d3 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:47.683453 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498d479e-0a9b-4009-956f-df75afa7c6f5 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:47.683853 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498d479e-0a9b-4009-956f-df75afa7c6f5 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:47.684269 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498d479e-0a9b-4009-956f-df75afa7c6f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:47.713836 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-498d479e-0a9b-4009-956f-df75afa7c6f5 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:47.713836 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-498d479e-0a9b-4009-956f-df75afa7c6f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:47.752236 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1553f4c0-38e0-425f-a34d-5d5c2659e950 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d81f597cb44c56a966537089bcce4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:47.753000 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1553f4c0-38e0-425f-a34d-5d5c2659e950 service cinder] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:47.753364 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1553f4c0-38e0-425f-a34d-5d5c2659e950 service cinder] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:47.753722 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1553f4c0-38e0-425f-a34d-5d5c2659e950 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:47.755673 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-1553f4c0-38e0-425f-a34d-5d5c2659e950 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:47.790620 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 921/1820] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:39:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:39:47.800924 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498d479e-0a9b-4009-956f-df75afa7c6f5 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:47.802750 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-498d479e-0a9b-4009-956f-df75afa7c6f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:47.832864 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1553f4c0-38e0-425f-a34d-5d5c2659e950 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:47.834611 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1553f4c0-38e0-425f-a34d-5d5c2659e950 service cinder] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:47.846076 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 900/1821] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:39:47 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:39:47.869689 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498d479e-0a9b-4009-956f-df75afa7c6f5 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:47.876524 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b383d4-a690-4696-886d-37ef8f7274f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:47.877523 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b383d4-a690-4696-886d-37ef8f7274f9 service neutron] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:47.877810 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b383d4-a690-4696-886d-37ef8f7274f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:47.878099 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b383d4-a690-4696-886d-37ef8f7274f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:47.890962 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b383d4-a690-4696-886d-37ef8f7274f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:47.893201 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b383d4-a690-4696-886d-37ef8f7274f9 service neutron] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:47.902829 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 922/1822] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:39:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 14:39:48.160954 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1553f4c0-38e0-425f-a34d-5d5c2659e950 service cinder] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:48.167750 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-1553f4c0-38e0-425f-a34d-5d5c2659e950 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:48.174833 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1553f4c0-38e0-425f-a34d-5d5c2659e950 service cinder] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:48.178693 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303d0506-9c27-4813-a803-49b8d8924941 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d81f597cb44c56a966537089bcce4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:48.179273 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303d0506-9c27-4813-a803-49b8d8924941 service cinder] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:48.179981 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303d0506-9c27-4813-a803-49b8d8924941 service cinder] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:48.179981 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303d0506-9c27-4813-a803-49b8d8924941 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:48.288959 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303d0506-9c27-4813-a803-49b8d8924941 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:48.291041 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303d0506-9c27-4813-a803-49b8d8924941 service cinder] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:48.303591 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 901/1823] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:39:48 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:39:48.796649 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b383d4-a690-4696-886d-37ef8f7274f9 service neutron] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:48.798444 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-b0b383d4-a690-4696-886d-37ef8f7274f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:48.819694 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b383d4-a690-4696-886d-37ef8f7274f9 service neutron] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:48.824934 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429131fa-3cc2-4051-94ac-24dd766629a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:48.826193 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429131fa-3cc2-4051-94ac-24dd766629a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:48.826499 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429131fa-3cc2-4051-94ac-24dd766629a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:48.826986 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429131fa-3cc2-4051-94ac-24dd766629a2 admin admin] PATH_INFO: `/v3/projects/c39871cb446c4a7c82d72754b2667e6a` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:48.829405 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429131fa-3cc2-4051-94ac-24dd766629a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c39871cb446c4a7c82d72754b2667e6a)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:48.834721 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429131fa-3cc2-4051-94ac-24dd766629a2 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:48.947461 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:39:49.005351 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-429131fa-3cc2-4051-94ac-24dd766629a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c39871cb446c4a7c82d72754b2667e6a', 'request_id': 'req-429131fa-3cc2-4051-94ac-24dd766629a2'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:49.083105 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-429131fa-3cc2-4051-94ac-24dd766629a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c39871cb446c4a7c82d72754b2667e6a', 'request_id': 'req-429131fa-3cc2-4051-94ac-24dd766629a2'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:49.089079 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-429131fa-3cc2-4051-94ac-24dd766629a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c39871cb446c4a7c82d72754b2667e6a', 'request_id': 'req-429131fa-3cc2-4051-94ac-24dd766629a2'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:49.100731 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-429131fa-3cc2-4051-94ac-24dd766629a2 admin admin] The token cache is being invalidate because project c39871cb446c4a7c82d72754b2667e6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:39:49.101015 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-429131fa-3cc2-4051-94ac-24dd766629a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:49.101822 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-429131fa-3cc2-4051-94ac-24dd766629a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:49.103394 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 923/1824] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:39:48 2022] DELETE /identity/v3/projects/c39871cb446c4a7c82d72754b2667e6a => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:39:49.107501 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303d0506-9c27-4813-a803-49b8d8924941 service cinder] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:49.117873 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303d0506-9c27-4813-a803-49b8d8924941 service cinder] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:49.174229 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce75246-a92c-44d8-b73d-c29e314b32d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:49.175112 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce75246-a92c-44d8-b73d-c29e314b32d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:49.175401 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce75246-a92c-44d8-b73d-c29e314b32d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:49.175679 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce75246-a92c-44d8-b73d-c29e314b32d0 admin admin] PATH_INFO: `/v3/users/de400d5a2f164b29980b8469a9de810f` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:49.176910 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce75246-a92c-44d8-b73d-c29e314b32d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de400d5a2f164b29980b8469a9de810f)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:49.178847 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce75246-a92c-44d8-b73d-c29e314b32d0 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:49.300222 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:39:49.441309 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-6ce75246-a92c-44d8-b73d-c29e314b32d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de400d5a2f164b29980b8469a9de810f'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:49.442338 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-6ce75246-a92c-44d8-b73d-c29e314b32d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de400d5a2f164b29980b8469a9de810f'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:49.443010 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-6ce75246-a92c-44d8-b73d-c29e314b32d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de400d5a2f164b29980b8469a9de810f'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:49.450300 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-6ce75246-a92c-44d8-b73d-c29e314b32d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de400d5a2f164b29980b8469a9de810f'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:49.462897 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-6ce75246-a92c-44d8-b73d-c29e314b32d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de400d5a2f164b29980b8469a9de810f'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:49.569849 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 902/1825] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:39:49 2022] DELETE /identity/v3/users/de400d5a2f164b29980b8469a9de810f => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:39:50.197485 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429131fa-3cc2-4051-94ac-24dd766629a2 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:50.199124 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-429131fa-3cc2-4051-94ac-24dd766629a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:50.296982 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429131fa-3cc2-4051-94ac-24dd766629a2 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:50.372135 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c606e4b-b5f1-4540-b557-c4c75145b45a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d81f597cb44c56a966537089bcce4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:50.372931 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c606e4b-b5f1-4540-b557-c4c75145b45a service cinder] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:50.373299 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c606e4b-b5f1-4540-b557-c4c75145b45a service cinder] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:50.373581 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c606e4b-b5f1-4540-b557-c4c75145b45a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:50.377459 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-1c606e4b-b5f1-4540-b557-c4c75145b45a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:50.494789 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c606e4b-b5f1-4540-b557-c4c75145b45a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:50.496258 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c606e4b-b5f1-4540-b557-c4c75145b45a service cinder] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:50.520997 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 924/1826] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:39:50 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:39:51.465503 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce75246-a92c-44d8-b73d-c29e314b32d0 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:51.468873 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-6ce75246-a92c-44d8-b73d-c29e314b32d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:51.607214 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce75246-a92c-44d8-b73d-c29e314b32d0 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:51.613185 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672d1565-3928-47c6-ae54-9089e0be17d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:51.614573 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672d1565-3928-47c6-ae54-9089e0be17d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:51.615088 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672d1565-3928-47c6-ae54-9089e0be17d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:51.615467 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672d1565-3928-47c6-ae54-9089e0be17d8 admin admin] PATH_INFO: `/v3/projects/3ac8b2d0929b496a9c183349e20910ce` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:51.616816 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672d1565-3928-47c6-ae54-9089e0be17d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ac8b2d0929b496a9c183349e20910ce)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:51.619215 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672d1565-3928-47c6-ae54-9089e0be17d8 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:51.754324 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:39:51.814706 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-672d1565-3928-47c6-ae54-9089e0be17d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ac8b2d0929b496a9c183349e20910ce', 'request_id': 'req-672d1565-3928-47c6-ae54-9089e0be17d8'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:51.950016 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-672d1565-3928-47c6-ae54-9089e0be17d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ac8b2d0929b496a9c183349e20910ce', 'request_id': 'req-672d1565-3928-47c6-ae54-9089e0be17d8'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:51.963933 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-672d1565-3928-47c6-ae54-9089e0be17d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ac8b2d0929b496a9c183349e20910ce', 'request_id': 'req-672d1565-3928-47c6-ae54-9089e0be17d8'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:51.976732 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-672d1565-3928-47c6-ae54-9089e0be17d8 admin admin] The token cache is being invalidate because project 3ac8b2d0929b496a9c183349e20910ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:39:51.977836 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-672d1565-3928-47c6-ae54-9089e0be17d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:51.977836 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-672d1565-3928-47c6-ae54-9089e0be17d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:39:51.983520 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 903/1827] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:39:51 2022] DELETE /identity/v3/projects/3ac8b2d0929b496a9c183349e20910ce => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:39:51.985401 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170612ba-edf0-4749-a22d-24f75b5d74d6 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:51.985866 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170612ba-edf0-4749-a22d-24f75b5d74d6 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:51.986306 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170612ba-edf0-4749-a22d-24f75b5d74d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:52.008143 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-170612ba-edf0-4749-a22d-24f75b5d74d6 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:52.009691 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-170612ba-edf0-4749-a22d-24f75b5d74d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:52.092556 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 925/1828] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:39:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:39:52.101417 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a363db-f2c7-41f1-8e42-fbeca5a6f78d None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:52.102098 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a363db-f2c7-41f1-8e42-fbeca5a6f78d None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:52.102557 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a363db-f2c7-41f1-8e42-fbeca5a6f78d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:52.139997 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-23a363db-f2c7-41f1-8e42-fbeca5a6f78d None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:52.142969 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-23a363db-f2c7-41f1-8e42-fbeca5a6f78d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:52.162214 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 904/1829] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:39:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:39:52.162339 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170612ba-edf0-4749-a22d-24f75b5d74d6 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:52.174696 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170612ba-edf0-4749-a22d-24f75b5d74d6 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:52.180223 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e214ca-91c0-46bc-8b9f-a8f606e66bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:52.181224 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e214ca-91c0-46bc-8b9f-a8f606e66bf2 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:52.181712 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e214ca-91c0-46bc-8b9f-a8f606e66bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:52.182060 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e214ca-91c0-46bc-8b9f-a8f606e66bf2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:52.182993 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e214ca-91c0-46bc-8b9f-a8f606e66bf2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:39:52.189475 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e214ca-91c0-46bc-8b9f-a8f606e66bf2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:52.189475 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e214ca-91c0-46bc-8b9f-a8f606e66bf2 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:52.207121 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 926/1830] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jun 7 14:39:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:39:52.207917 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a363db-f2c7-41f1-8e42-fbeca5a6f78d None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:52.219981 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a363db-f2c7-41f1-8e42-fbeca5a6f78d None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:52.230480 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89405e8c-09f4-4d4e-8abe-46fea43a5e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:52.231395 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89405e8c-09f4-4d4e-8abe-46fea43a5e49 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:52.231689 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89405e8c-09f4-4d4e-8abe-46fea43a5e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:52.235183 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89405e8c-09f4-4d4e-8abe-46fea43a5e49 admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:52.236083 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89405e8c-09f4-4d4e-8abe-46fea43a5e49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:52.238462 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89405e8c-09f4-4d4e-8abe-46fea43a5e49 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:52.354834 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 905/1831] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:39:52 2022] POST /identity/v3/projects => generated 361 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:39:52.359457 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e214ca-91c0-46bc-8b9f-a8f606e66bf2 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:52.393212 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e214ca-91c0-46bc-8b9f-a8f606e66bf2 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:52.510358 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8130ad-a593-41f8-98ce-780381bacea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:52.511141 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8130ad-a593-41f8-98ce-780381bacea1 admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:52.511648 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8130ad-a593-41f8-98ce-780381bacea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:52.512399 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8130ad-a593-41f8-98ce-780381bacea1 admin admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:52.513289 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8130ad-a593-41f8-98ce-780381bacea1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:52.516189 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8130ad-a593-41f8-98ce-780381bacea1 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:52.670785 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 927/1832] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:39:52 2022] POST /identity/v3/users => generated 281 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:39:52.672615 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89405e8c-09f4-4d4e-8abe-46fea43a5e49 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:52.689786 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89405e8c-09f4-4d4e-8abe-46fea43a5e49 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:52.695235 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c75c71e-6af4-4b76-a7c2-0d39ee62ad4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:52.696049 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c75c71e-6af4-4b76-a7c2-0d39ee62ad4b admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:52.696341 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c75c71e-6af4-4b76-a7c2-0d39ee62ad4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:52.696678 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c75c71e-6af4-4b76-a7c2-0d39ee62ad4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:52.697446 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c75c71e-6af4-4b76-a7c2-0d39ee62ad4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:52.699114 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c75c71e-6af4-4b76-a7c2-0d39ee62ad4b admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:52.711501 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 906/1833] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:39:52 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:39:52.715834 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8130ad-a593-41f8-98ce-780381bacea1 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:52.726609 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8130ad-a593-41f8-98ce-780381bacea1 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:52.732947 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e905480-0790-48f7-9b4c-8643befddd04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:52.733770 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e905480-0790-48f7-9b4c-8643befddd04 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:52.734147 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e905480-0790-48f7-9b4c-8643befddd04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:52.734495 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e905480-0790-48f7-9b4c-8643befddd04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:52.735246 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e905480-0790-48f7-9b4c-8643befddd04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:52.736900 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e905480-0790-48f7-9b4c-8643befddd04 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:52.749629 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 928/1834] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:39:52 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:39:52.758419 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c75c71e-6af4-4b76-a7c2-0d39ee62ad4b admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:52.769460 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c75c71e-6af4-4b76-a7c2-0d39ee62ad4b admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:52.781691 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08bc8d3-c2e8-4433-8692-95ac22e2fc7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:52.782994 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08bc8d3-c2e8-4433-8692-95ac22e2fc7a admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:52.783280 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08bc8d3-c2e8-4433-8692-95ac22e2fc7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:52.783553 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08bc8d3-c2e8-4433-8692-95ac22e2fc7a admin admin] PATH_INFO: `/v3/projects/658773179d67451cb84389ae70cc3da4/users/33b79181a2b74e369d89e47793fee642/roles/3771341b3be641139e752aab690397a4` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:52.803895 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08bc8d3-c2e8-4433-8692-95ac22e2fc7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=658773179d67451cb84389ae70cc3da4, user_id=33b79181a2b74e369d89e47793fee642, role_id=3771341b3be641139e752aab690397a4)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:52.807199 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08bc8d3-c2e8-4433-8692-95ac22e2fc7a admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:52.917667 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 907/1835] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:39:52 2022] PUT /identity/v3/projects/658773179d67451cb84389ae70cc3da4/users/33b79181a2b74e369d89e47793fee642/roles/3771341b3be641139e752aab690397a4 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:39:52.932142 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef40923-8915-413f-babe-f21d7b7e63a3 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:52.934984 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef40923-8915-413f-babe-f21d7b7e63a3 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:52.935422 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef40923-8915-413f-babe-f21d7b7e63a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:52.976151 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-eef40923-8915-413f-babe-f21d7b7e63a3 None None] MFA Rules not processed for user `33b79181a2b74e369d89e47793fee642`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:52.978003 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-eef40923-8915-413f-babe-f21d7b7e63a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:53.051797 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 929/1836] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:39:52 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:39:53.054907 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:39:53.060329 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08bc8d3-c2e8-4433-8692-95ac22e2fc7a admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:53.130051 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08bc8d3-c2e8-4433-8692-95ac22e2fc7a admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:53.135238 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83816667-e5dd-4d92-ae34-353da7dbb388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:53.135999 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83816667-e5dd-4d92-ae34-353da7dbb388 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:53.136353 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83816667-e5dd-4d92-ae34-353da7dbb388 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:53.137450 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83816667-e5dd-4d92-ae34-353da7dbb388 admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:53.137943 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83816667-e5dd-4d92-ae34-353da7dbb388 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:53.140084 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83816667-e5dd-4d92-ae34-353da7dbb388 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:53.276066 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 908/1837] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:39:53 2022] POST /identity/v3/projects => generated 365 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:39:53.280689 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef40923-8915-413f-babe-f21d7b7e63a3 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:53.295395 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef40923-8915-413f-babe-f21d7b7e63a3 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:53.367353 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe61a10-f43c-4446-a527-263c93c7d791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:53.368582 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe61a10-f43c-4446-a527-263c93c7d791 admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:53.369156 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe61a10-f43c-4446-a527-263c93c7d791 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:53.369451 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe61a10-f43c-4446-a527-263c93c7d791 admin admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:53.370176 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe61a10-f43c-4446-a527-263c93c7d791 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:53.372112 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe61a10-f43c-4446-a527-263c93c7d791 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:53.455921 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 930/1838] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:39:53 2022] POST /identity/v3/users => generated 289 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:39:53.462686 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83816667-e5dd-4d92-ae34-353da7dbb388 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:53.474776 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83816667-e5dd-4d92-ae34-353da7dbb388 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:53.479466 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7bc340-4830-4ebd-b68d-1dc48cdd45dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:53.480141 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7bc340-4830-4ebd-b68d-1dc48cdd45dd admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:53.480460 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7bc340-4830-4ebd-b68d-1dc48cdd45dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:53.480746 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7bc340-4830-4ebd-b68d-1dc48cdd45dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:53.481571 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7bc340-4830-4ebd-b68d-1dc48cdd45dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:53.483842 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7bc340-4830-4ebd-b68d-1dc48cdd45dd admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:53.494337 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 909/1839] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:39:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:39:53.498564 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe61a10-f43c-4446-a527-263c93c7d791 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:53.508868 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe61a10-f43c-4446-a527-263c93c7d791 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:53.513775 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4479a09d-1a67-4994-a978-c10ed1c6d29a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:53.517328 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4479a09d-1a67-4994-a978-c10ed1c6d29a admin admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:53.517328 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4479a09d-1a67-4994-a978-c10ed1c6d29a admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:53.517328 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4479a09d-1a67-4994-a978-c10ed1c6d29a admin admin] PATH_INFO: `/v3/projects/426e712d386d41b888248c3d3a7f7ef4/users/33d3770cc345427e99cacff5d9b1a198/roles/9baff2b18f2649f7b757ff2e1f63c853` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:53.533365 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4479a09d-1a67-4994-a978-c10ed1c6d29a admin admin] RBAC: Authorizing `identity:create_grant(project_id=426e712d386d41b888248c3d3a7f7ef4, user_id=33d3770cc345427e99cacff5d9b1a198, role_id=9baff2b18f2649f7b757ff2e1f63c853)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:53.536460 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4479a09d-1a67-4994-a978-c10ed1c6d29a admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:53.649365 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 931/1840] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:39:53 2022] PUT /identity/v3/projects/426e712d386d41b888248c3d3a7f7ef4/users/33d3770cc345427e99cacff5d9b1a198/roles/9baff2b18f2649f7b757ff2e1f63c853 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:39:53.653575 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7bc340-4830-4ebd-b68d-1dc48cdd45dd admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:53.664334 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7bc340-4830-4ebd-b68d-1dc48cdd45dd admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:53.724903 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd4081b-6fa2-40c6-abff-af268db5cd49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:53.726548 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd4081b-6fa2-40c6-abff-af268db5cd49 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:53.727079 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd4081b-6fa2-40c6-abff-af268db5cd49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:53.727358 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd4081b-6fa2-40c6-abff-af268db5cd49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:53.728105 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd4081b-6fa2-40c6-abff-af268db5cd49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:53.729774 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd4081b-6fa2-40c6-abff-af268db5cd49 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:53.741188 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 910/1841] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:39:53 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:39:53.745595 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4479a09d-1a67-4994-a978-c10ed1c6d29a admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:53.756884 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4479a09d-1a67-4994-a978-c10ed1c6d29a admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:53.763124 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6f1dc1-e70b-4a53-a82e-d91ca1208792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:53.764037 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6f1dc1-e70b-4a53-a82e-d91ca1208792 admin admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:53.764442 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6f1dc1-e70b-4a53-a82e-d91ca1208792 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:53.764720 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6f1dc1-e70b-4a53-a82e-d91ca1208792 admin admin] PATH_INFO: `/v3/projects/426e712d386d41b888248c3d3a7f7ef4/users/33d3770cc345427e99cacff5d9b1a198/roles/9baff2b18f2649f7b757ff2e1f63c853` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:53.766755 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6f1dc1-e70b-4a53-a82e-d91ca1208792 admin admin] RBAC: Authorizing `identity:create_grant(project_id=426e712d386d41b888248c3d3a7f7ef4, user_id=33d3770cc345427e99cacff5d9b1a198, role_id=9baff2b18f2649f7b757ff2e1f63c853)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:53.770162 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6f1dc1-e70b-4a53-a82e-d91ca1208792 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:53.783016 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 932/1842] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:39:53 2022] PUT /identity/v3/projects/426e712d386d41b888248c3d3a7f7ef4/users/33d3770cc345427e99cacff5d9b1a198/roles/9baff2b18f2649f7b757ff2e1f63c853 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:39:53.793312 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc5ba9a-d342-4f81-a7ba-51f3f39910ca None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:53.793608 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc5ba9a-d342-4f81-a7ba-51f3f39910ca None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:53.793891 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc5ba9a-d342-4f81-a7ba-51f3f39910ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:53.825164 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-3dc5ba9a-d342-4f81-a7ba-51f3f39910ca None None] MFA Rules not processed for user `33d3770cc345427e99cacff5d9b1a198`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:53.826405 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-3dc5ba9a-d342-4f81-a7ba-51f3f39910ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:53.898792 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 911/1843] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:39:53 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:39:53.911209 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6f1dc1-e70b-4a53-a82e-d91ca1208792 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:53.913063 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-1e6f1dc1-e70b-4a53-a82e-d91ca1208792 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:53.980799 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6f1dc1-e70b-4a53-a82e-d91ca1208792 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:53.984724 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa60c40-0b8f-41ec-821b-3b1d64921cf0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d81f597cb44c56a966537089bcce4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:53.985380 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa60c40-0b8f-41ec-821b-3b1d64921cf0 service cinder] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:53.985663 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa60c40-0b8f-41ec-821b-3b1d64921cf0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:53.985994 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa60c40-0b8f-41ec-821b-3b1d64921cf0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:53.995452 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa60c40-0b8f-41ec-821b-3b1d64921cf0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:53.996948 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa60c40-0b8f-41ec-821b-3b1d64921cf0 service cinder] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:54.006288 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 933/1844] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:39:53 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:39:54.232033 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc5ba9a-d342-4f81-a7ba-51f3f39910ca None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:54.233462 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-3dc5ba9a-d342-4f81-a7ba-51f3f39910ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:54.245655 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc5ba9a-d342-4f81-a7ba-51f3f39910ca None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:54.249848 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c88f747-feee-45db-8d77-93ef47410352 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d81f597cb44c56a966537089bcce4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:54.250537 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c88f747-feee-45db-8d77-93ef47410352 service cinder] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:54.250821 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c88f747-feee-45db-8d77-93ef47410352 service cinder] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:54.251098 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c88f747-feee-45db-8d77-93ef47410352 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:54.320764 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c88f747-feee-45db-8d77-93ef47410352 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:54.322265 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c88f747-feee-45db-8d77-93ef47410352 service cinder] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:54.341502 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 912/1845] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:39:54 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:39:54.426244 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:39:56.106301 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:39:56.157374 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:39:56.683280 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:39:56.894807 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:39:57.122788 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:39:57.267003 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:39:57.620590 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:39:57.724869 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:39:58.151956 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:39:58.393663 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf56dc2-0b3a-4151-b6fd-5b943cc2368e None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:58.394187 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf56dc2-0b3a-4151-b6fd-5b943cc2368e None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:58.394690 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf56dc2-0b3a-4151-b6fd-5b943cc2368e None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:58.396709 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 934/1846] 10.222.0.85 () {54 vars in 921 bytes} [Tue Jun 7 14:39:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:39:58.406252 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1082689c-418d-4236-b7a0-18381a33525a None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:58.406810 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1082689c-418d-4236-b7a0-18381a33525a None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:58.411554 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1082689c-418d-4236-b7a0-18381a33525a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:58.441305 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-1082689c-418d-4236-b7a0-18381a33525a None None] MFA Rules not processed for user `8e857db4902e42989d0d040bbc8517c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:58.442753 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-1082689c-418d-4236-b7a0-18381a33525a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:58.560577 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 913/1847] 10.222.0.85 () {58 vars in 1018 bytes} [Tue Jun 7 14:39:58 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:39:58.579969 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf56dc2-0b3a-4151-b6fd-5b943cc2368e None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:58.583573 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-9cf56dc2-0b3a-4151-b6fd-5b943cc2368e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:58.608752 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf56dc2-0b3a-4151-b6fd-5b943cc2368e None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:58.617350 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1149c76f-6e2f-4ec5-bc40-2308951d5ae5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e857db4902e42989d0d040bbc8517c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:58.620276 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1149c76f-6e2f-4ec5-bc40-2308951d5ae5 service nova] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:58.620868 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1149c76f-6e2f-4ec5-bc40-2308951d5ae5 service nova] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:58.621477 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1149c76f-6e2f-4ec5-bc40-2308951d5ae5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:58.635663 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1149c76f-6e2f-4ec5-bc40-2308951d5ae5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:58.637724 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1149c76f-6e2f-4ec5-bc40-2308951d5ae5 service nova] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:58.645949 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:39:58.659371 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 935/1848] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:39:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:39:58.838540 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:39:59.397508 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c3754c-a26d-4c2c-8fd0-f55887bb1838 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:59.398177 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c3754c-a26d-4c2c-8fd0-f55887bb1838 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:59.398656 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c3754c-a26d-4c2c-8fd0-f55887bb1838 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:59.402719 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 914/1849] 10.222.0.85 () {54 vars in 921 bytes} [Tue Jun 7 14:39:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:39:59.432445 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dcc1a6-45a2-4324-8b69-6cb550e3906c None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:59.432958 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dcc1a6-45a2-4324-8b69-6cb550e3906c None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:59.433032 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dcc1a6-45a2-4324-8b69-6cb550e3906c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:59.472710 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-35dcc1a6-45a2-4324-8b69-6cb550e3906c None None] MFA Rules not processed for user `8e857db4902e42989d0d040bbc8517c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:39:59.474537 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-35dcc1a6-45a2-4324-8b69-6cb550e3906c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:39:59.488124 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 936/1850] 10.222.0.85 () {58 vars in 1018 bytes} [Tue Jun 7 14:39:59 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:39:59.566963 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:39:59.691411 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c3754c-a26d-4c2c-8fd0-f55887bb1838 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:39:59.752428 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c3754c-a26d-4c2c-8fd0-f55887bb1838 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:39:59.780888 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f1ffa4-930f-4b8c-b394-e81d5ef41dca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e857db4902e42989d0d040bbc8517c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:39:59.782021 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f1ffa4-930f-4b8c-b394-e81d5ef41dca service nova] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:39:59.782535 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f1ffa4-930f-4b8c-b394-e81d5ef41dca service nova] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:39:59.783156 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f1ffa4-930f-4b8c-b394-e81d5ef41dca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:39:59.821691 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f1ffa4-930f-4b8c-b394-e81d5ef41dca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:39:59.825878 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f1ffa4-930f-4b8c-b394-e81d5ef41dca service nova] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:39:59.852602 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 915/1851] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:39:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:40:01.290621 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:40:01.820584 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dcc1a6-45a2-4324-8b69-6cb550e3906c None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:01.834225 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:40:01.899942 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dcc1a6-45a2-4324-8b69-6cb550e3906c None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:01.904616 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e97a236-bb6c-498f-89b8-146f8934e252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:01.906058 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e97a236-bb6c-498f-89b8-146f8934e252 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:01.906395 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e97a236-bb6c-498f-89b8-146f8934e252 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:01.908005 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e97a236-bb6c-498f-89b8-146f8934e252 admin admin] PATH_INFO: `/v3/users/33b79181a2b74e369d89e47793fee642` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:01.908720 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e97a236-bb6c-498f-89b8-146f8934e252 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33b79181a2b74e369d89e47793fee642)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:01.910707 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e97a236-bb6c-498f-89b8-146f8934e252 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:02.154496 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-1e97a236-bb6c-498f-89b8-146f8934e252 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33b79181a2b74e369d89e47793fee642'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:02.155757 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-1e97a236-bb6c-498f-89b8-146f8934e252 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33b79181a2b74e369d89e47793fee642'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:02.156464 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-1e97a236-bb6c-498f-89b8-146f8934e252 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33b79181a2b74e369d89e47793fee642'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:02.163528 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-1e97a236-bb6c-498f-89b8-146f8934e252 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33b79181a2b74e369d89e47793fee642'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:02.175899 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-1e97a236-bb6c-498f-89b8-146f8934e252 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33b79181a2b74e369d89e47793fee642'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:02.253541 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 937/1852] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:40:01 2022] DELETE /identity/v3/users/33b79181a2b74e369d89e47793fee642 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:40:02.258035 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544d24e4-875a-4d5e-8085-6ffd686215c7 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:02.258322 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544d24e4-875a-4d5e-8085-6ffd686215c7 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:02.258601 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544d24e4-875a-4d5e-8085-6ffd686215c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:02.276916 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-544d24e4-875a-4d5e-8085-6ffd686215c7 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:40:02.278159 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-544d24e4-875a-4d5e-8085-6ffd686215c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:02.336501 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 916/1853] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:40:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:40:02.346706 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e97a236-bb6c-498f-89b8-146f8934e252 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:02.348133 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-1e97a236-bb6c-498f-89b8-146f8934e252 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:02.410845 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:40:02.415964 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e97a236-bb6c-498f-89b8-146f8934e252 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:02.421107 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289dbba4-fc2e-4a39-b226-7adfd8627269 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:02.421547 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289dbba4-fc2e-4a39-b226-7adfd8627269 service neutron] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:02.421832 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289dbba4-fc2e-4a39-b226-7adfd8627269 service neutron] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:02.422110 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289dbba4-fc2e-4a39-b226-7adfd8627269 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:02.432447 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289dbba4-fc2e-4a39-b226-7adfd8627269 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:02.433926 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289dbba4-fc2e-4a39-b226-7adfd8627269 service neutron] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:02.444533 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 938/1854] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:40:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:40:02.809863 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544d24e4-875a-4d5e-8085-6ffd686215c7 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:02.812159 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-544d24e4-875a-4d5e-8085-6ffd686215c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:02.921263 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544d24e4-875a-4d5e-8085-6ffd686215c7 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:02.924974 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0a9cf4-35a8-4d22-bc57-0ed59cdc09e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d81f597cb44c56a966537089bcce4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:02.925721 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0a9cf4-35a8-4d22-bc57-0ed59cdc09e6 service cinder] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:02.926062 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0a9cf4-35a8-4d22-bc57-0ed59cdc09e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:02.927636 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0a9cf4-35a8-4d22-bc57-0ed59cdc09e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:02.932331 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-3b0a9cf4-35a8-4d22-bc57-0ed59cdc09e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:03.015843 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0a9cf4-35a8-4d22-bc57-0ed59cdc09e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:03.017449 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0a9cf4-35a8-4d22-bc57-0ed59cdc09e6 service cinder] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:03.027420 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 917/1855] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:40:02 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:40:03.094726 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289dbba4-fc2e-4a39-b226-7adfd8627269 service neutron] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:03.097743 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-289dbba4-fc2e-4a39-b226-7adfd8627269 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:03.114237 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289dbba4-fc2e-4a39-b226-7adfd8627269 service neutron] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:03.133596 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf1e061-45a2-469e-8be0-36456829e754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:03.135028 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf1e061-45a2-469e-8be0-36456829e754 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:03.135779 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf1e061-45a2-469e-8be0-36456829e754 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:03.136407 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf1e061-45a2-469e-8be0-36456829e754 admin admin] PATH_INFO: `/v3/projects/658773179d67451cb84389ae70cc3da4` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:03.138405 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf1e061-45a2-469e-8be0-36456829e754 admin admin] RBAC: Authorizing `identity:delete_project(project_id=658773179d67451cb84389ae70cc3da4)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:03.140611 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf1e061-45a2-469e-8be0-36456829e754 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:03.178169 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:40:03.312062 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-fbf1e061-45a2-469e-8be0-36456829e754 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '658773179d67451cb84389ae70cc3da4', 'request_id': 'req-fbf1e061-45a2-469e-8be0-36456829e754'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:03.403505 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-fbf1e061-45a2-469e-8be0-36456829e754 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '658773179d67451cb84389ae70cc3da4', 'request_id': 'req-fbf1e061-45a2-469e-8be0-36456829e754'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:03.409949 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-fbf1e061-45a2-469e-8be0-36456829e754 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '658773179d67451cb84389ae70cc3da4', 'request_id': 'req-fbf1e061-45a2-469e-8be0-36456829e754'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:03.421635 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-fbf1e061-45a2-469e-8be0-36456829e754 admin admin] The token cache is being invalidate because project 658773179d67451cb84389ae70cc3da4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:40:03.421859 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-fbf1e061-45a2-469e-8be0-36456829e754 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:03.422506 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-fbf1e061-45a2-469e-8be0-36456829e754 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:03.424098 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 939/1856] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:40:03 2022] DELETE /identity/v3/projects/658773179d67451cb84389ae70cc3da4 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:40:03.428252 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0a9cf4-35a8-4d22-bc57-0ed59cdc09e6 service cinder] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:03.441628 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0a9cf4-35a8-4d22-bc57-0ed59cdc09e6 service cinder] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:03.446575 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:40:03.505654 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd9329e-bf8c-4b73-84ab-f53144a43d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:03.506540 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd9329e-bf8c-4b73-84ab-f53144a43d51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:03.506824 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd9329e-bf8c-4b73-84ab-f53144a43d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:03.507104 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd9329e-bf8c-4b73-84ab-f53144a43d51 admin admin] PATH_INFO: `/v3/users/33d3770cc345427e99cacff5d9b1a198` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:03.508302 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd9329e-bf8c-4b73-84ab-f53144a43d51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33d3770cc345427e99cacff5d9b1a198)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:03.510252 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd9329e-bf8c-4b73-84ab-f53144a43d51 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:03.728806 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-0fd9329e-bf8c-4b73-84ab-f53144a43d51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33d3770cc345427e99cacff5d9b1a198'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:03.729564 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-0fd9329e-bf8c-4b73-84ab-f53144a43d51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33d3770cc345427e99cacff5d9b1a198'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:03.730414 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-0fd9329e-bf8c-4b73-84ab-f53144a43d51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33d3770cc345427e99cacff5d9b1a198'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:03.740031 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-0fd9329e-bf8c-4b73-84ab-f53144a43d51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33d3770cc345427e99cacff5d9b1a198'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:03.753218 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-0fd9329e-bf8c-4b73-84ab-f53144a43d51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33d3770cc345427e99cacff5d9b1a198'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:03.877534 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 918/1857] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:40:03 2022] DELETE /identity/v3/users/33d3770cc345427e99cacff5d9b1a198 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:40:04.596836 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf1e061-45a2-469e-8be0-36456829e754 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:04.600297 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-fbf1e061-45a2-469e-8be0-36456829e754 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:04.625027 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf1e061-45a2-469e-8be0-36456829e754 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:04.846671 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0971b6b0-f151-4cf4-8afe-4e2fff076ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:04.846671 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0971b6b0-f151-4cf4-8afe-4e2fff076ece admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:04.846671 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0971b6b0-f151-4cf4-8afe-4e2fff076ece admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:04.846671 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0971b6b0-f151-4cf4-8afe-4e2fff076ece admin admin] PATH_INFO: `/v3/projects/426e712d386d41b888248c3d3a7f7ef4` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:04.847549 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0971b6b0-f151-4cf4-8afe-4e2fff076ece admin admin] RBAC: Authorizing `identity:delete_project(project_id=426e712d386d41b888248c3d3a7f7ef4)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:04.847549 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0971b6b0-f151-4cf4-8afe-4e2fff076ece admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:05.029584 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-0971b6b0-f151-4cf4-8afe-4e2fff076ece admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '426e712d386d41b888248c3d3a7f7ef4', 'request_id': 'req-0971b6b0-f151-4cf4-8afe-4e2fff076ece'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:05.111579 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-0971b6b0-f151-4cf4-8afe-4e2fff076ece admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '426e712d386d41b888248c3d3a7f7ef4', 'request_id': 'req-0971b6b0-f151-4cf4-8afe-4e2fff076ece'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:05.124281 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-0971b6b0-f151-4cf4-8afe-4e2fff076ece admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '426e712d386d41b888248c3d3a7f7ef4', 'request_id': 'req-0971b6b0-f151-4cf4-8afe-4e2fff076ece'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:05.165941 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-0971b6b0-f151-4cf4-8afe-4e2fff076ece admin admin] The token cache is being invalidate because project 426e712d386d41b888248c3d3a7f7ef4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:40:05.168836 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-0971b6b0-f151-4cf4-8afe-4e2fff076ece admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:05.169508 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-0971b6b0-f151-4cf4-8afe-4e2fff076ece admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:05.174219 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 940/1858] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:40:04 2022] DELETE /identity/v3/projects/426e712d386d41b888248c3d3a7f7ef4 => generated 0 bytes in 578 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:40:05.180173 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4688761-15f0-4a94-8ab4-794e7fc379bf None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:05.181141 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4688761-15f0-4a94-8ab4-794e7fc379bf None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:05.182763 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4688761-15f0-4a94-8ab4-794e7fc379bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:05.230525 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-f4688761-15f0-4a94-8ab4-794e7fc379bf None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:40:05.234641 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-f4688761-15f0-4a94-8ab4-794e7fc379bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:05.341701 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 919/1859] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:40:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:40:05.344880 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84731957-2682-43bd-bae5-6f7a8a71c5f7 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:05.345189 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84731957-2682-43bd-bae5-6f7a8a71c5f7 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:05.345491 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84731957-2682-43bd-bae5-6f7a8a71c5f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:05.377489 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-84731957-2682-43bd-bae5-6f7a8a71c5f7 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:40:05.378936 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-84731957-2682-43bd-bae5-6f7a8a71c5f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:05.390237 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 941/1860] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:40:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:40:05.394665 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4688761-15f0-4a94-8ab4-794e7fc379bf None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:05.407256 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4688761-15f0-4a94-8ab4-794e7fc379bf None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:05.412907 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496a11eb-c87c-4e0b-8f6d-7cc09654072f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:05.413588 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a11eb-c87c-4e0b-8f6d-7cc09654072f admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:05.414599 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a11eb-c87c-4e0b-8f6d-7cc09654072f admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:05.414893 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a11eb-c87c-4e0b-8f6d-7cc09654072f admin admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:05.415842 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496a11eb-c87c-4e0b-8f6d-7cc09654072f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:40:05.416304 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496a11eb-c87c-4e0b-8f6d-7cc09654072f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:05.422602 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496a11eb-c87c-4e0b-8f6d-7cc09654072f admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:05.444408 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 920/1861] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jun 7 14:40:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:40:05.447235 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84731957-2682-43bd-bae5-6f7a8a71c5f7 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:05.460574 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84731957-2682-43bd-bae5-6f7a8a71c5f7 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:05.462781 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d34180-4a0b-47c9-bc26-06ed76b92b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:05.463461 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d34180-4a0b-47c9-bc26-06ed76b92b7a admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:05.463739 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d34180-4a0b-47c9-bc26-06ed76b92b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:05.464023 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d34180-4a0b-47c9-bc26-06ed76b92b7a admin admin] PATH_INFO: `/v3/projects` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:05.464770 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d34180-4a0b-47c9-bc26-06ed76b92b7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:05.466678 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d34180-4a0b-47c9-bc26-06ed76b92b7a admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:05.585071 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 942/1862] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:40:05 2022] POST /identity/v3/projects => generated 385 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:40:05.590025 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496a11eb-c87c-4e0b-8f6d-7cc09654072f admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:05.609349 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496a11eb-c87c-4e0b-8f6d-7cc09654072f admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:05.628036 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:40:05.687856 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e96f8d5-ee76-4d8a-8301-ca98ae291558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:05.688562 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e96f8d5-ee76-4d8a-8301-ca98ae291558 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:05.688851 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e96f8d5-ee76-4d8a-8301-ca98ae291558 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:05.689130 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e96f8d5-ee76-4d8a-8301-ca98ae291558 admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:05.689829 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e96f8d5-ee76-4d8a-8301-ca98ae291558 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:05.692935 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e96f8d5-ee76-4d8a-8301-ca98ae291558 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:05.822208 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 921/1863] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:40:05 2022] POST /identity/v3/users => generated 293 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:40:05.823493 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d34180-4a0b-47c9-bc26-06ed76b92b7a admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:05.835396 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d34180-4a0b-47c9-bc26-06ed76b92b7a admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:05.840221 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11ee92d-db62-40f8-aea2-8e7bee0a3232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:05.840922 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11ee92d-db62-40f8-aea2-8e7bee0a3232 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:05.841258 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11ee92d-db62-40f8-aea2-8e7bee0a3232 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:05.841540 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11ee92d-db62-40f8-aea2-8e7bee0a3232 admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:05.842270 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11ee92d-db62-40f8-aea2-8e7bee0a3232 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:05.843872 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11ee92d-db62-40f8-aea2-8e7bee0a3232 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:05.845984 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:40:05.855481 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 943/1864] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:40:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 14:40:05.860709 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e96f8d5-ee76-4d8a-8301-ca98ae291558 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:05.871037 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e96f8d5-ee76-4d8a-8301-ca98ae291558 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:05.876537 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbe46f2-d0d4-4929-8c9a-7cd224ab0214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:05.877205 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbe46f2-d0d4-4929-8c9a-7cd224ab0214 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:05.877492 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbe46f2-d0d4-4929-8c9a-7cd224ab0214 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:05.877773 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbe46f2-d0d4-4929-8c9a-7cd224ab0214 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:05.878507 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbe46f2-d0d4-4929-8c9a-7cd224ab0214 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:05.880100 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbe46f2-d0d4-4929-8c9a-7cd224ab0214 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:05.892229 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 922/1865] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:40:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:40:05.896582 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11ee92d-db62-40f8-aea2-8e7bee0a3232 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:05.906687 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11ee92d-db62-40f8-aea2-8e7bee0a3232 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:05.911339 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c64e82-8ffb-436b-a2a8-dc3d8eab4b77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:05.912220 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c64e82-8ffb-436b-a2a8-dc3d8eab4b77 admin admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:05.921591 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c64e82-8ffb-436b-a2a8-dc3d8eab4b77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:05.921591 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c64e82-8ffb-436b-a2a8-dc3d8eab4b77 admin admin] PATH_INFO: `/v3/projects/33519627a442485490b84fd2a5c1c8ee/users/a0f8ad0379cf48e7b821d12171382613/roles/3771341b3be641139e752aab690397a4` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:05.927300 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c64e82-8ffb-436b-a2a8-dc3d8eab4b77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33519627a442485490b84fd2a5c1c8ee, user_id=a0f8ad0379cf48e7b821d12171382613, role_id=3771341b3be641139e752aab690397a4)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:05.929974 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c64e82-8ffb-436b-a2a8-dc3d8eab4b77 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:06.020123 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 944/1866] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:40:05 2022] PUT /identity/v3/projects/33519627a442485490b84fd2a5c1c8ee/users/a0f8ad0379cf48e7b821d12171382613/roles/3771341b3be641139e752aab690397a4 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:40:06.030589 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf78a142-3a64-4c72-aad0-ad277308bfa2 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:06.030938 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf78a142-3a64-4c72-aad0-ad277308bfa2 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:06.031252 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf78a142-3a64-4c72-aad0-ad277308bfa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:06.057726 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-bf78a142-3a64-4c72-aad0-ad277308bfa2 None None] MFA Rules not processed for user `a0f8ad0379cf48e7b821d12171382613`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:40:06.059613 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-bf78a142-3a64-4c72-aad0-ad277308bfa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:06.142047 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 923/1867] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:40:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:40:06.146297 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c64e82-8ffb-436b-a2a8-dc3d8eab4b77 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:06.255875 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c64e82-8ffb-436b-a2a8-dc3d8eab4b77 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:06.261762 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00760c9-1220-454a-86a4-5eaa6d33cf48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:06.262882 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00760c9-1220-454a-86a4-5eaa6d33cf48 admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:06.263565 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00760c9-1220-454a-86a4-5eaa6d33cf48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:06.264151 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00760c9-1220-454a-86a4-5eaa6d33cf48 admin admin] PATH_INFO: `/v3/projects` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:06.265286 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00760c9-1220-454a-86a4-5eaa6d33cf48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:06.267794 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00760c9-1220-454a-86a4-5eaa6d33cf48 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:06.454371 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 945/1868] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:40:06 2022] POST /identity/v3/projects => generated 385 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:40:06.458036 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf78a142-3a64-4c72-aad0-ad277308bfa2 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:06.472226 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf78a142-3a64-4c72-aad0-ad277308bfa2 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:06.538101 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551121bf-c581-49c9-b119-bd3de527e278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:06.539094 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551121bf-c581-49c9-b119-bd3de527e278 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:06.539455 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551121bf-c581-49c9-b119-bd3de527e278 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:06.539824 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551121bf-c581-49c9-b119-bd3de527e278 admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:06.540628 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551121bf-c581-49c9-b119-bd3de527e278 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:06.542948 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551121bf-c581-49c9-b119-bd3de527e278 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:06.684730 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 924/1869] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:40:06 2022] POST /identity/v3/users => generated 299 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:40:06.691237 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00760c9-1220-454a-86a4-5eaa6d33cf48 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:06.703980 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00760c9-1220-454a-86a4-5eaa6d33cf48 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:06.717962 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f72c0a1-1e57-4f09-8e13-93de7414fa67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:06.719157 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f72c0a1-1e57-4f09-8e13-93de7414fa67 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:06.719675 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f72c0a1-1e57-4f09-8e13-93de7414fa67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:06.720252 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f72c0a1-1e57-4f09-8e13-93de7414fa67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:06.723568 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f72c0a1-1e57-4f09-8e13-93de7414fa67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:06.725925 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f72c0a1-1e57-4f09-8e13-93de7414fa67 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:06.741952 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 946/1870] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:40:06 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:40:06.745778 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551121bf-c581-49c9-b119-bd3de527e278 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:06.758885 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551121bf-c581-49c9-b119-bd3de527e278 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:06.775382 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf69561-0e38-45fe-a3d2-39c3501ac9cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:06.775382 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf69561-0e38-45fe-a3d2-39c3501ac9cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:06.775382 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf69561-0e38-45fe-a3d2-39c3501ac9cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:06.775382 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf69561-0e38-45fe-a3d2-39c3501ac9cc admin admin] PATH_INFO: `/v3/projects/4d77f0659785486cbf904c5269901eb6/users/b4a3dcf0c791421796d741856a1bc486/roles/9baff2b18f2649f7b757ff2e1f63c853` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:06.787210 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf69561-0e38-45fe-a3d2-39c3501ac9cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d77f0659785486cbf904c5269901eb6, user_id=b4a3dcf0c791421796d741856a1bc486, role_id=9baff2b18f2649f7b757ff2e1f63c853)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:06.790293 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf69561-0e38-45fe-a3d2-39c3501ac9cc admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:06.913485 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 925/1871] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:40:06 2022] PUT /identity/v3/projects/4d77f0659785486cbf904c5269901eb6/users/b4a3dcf0c791421796d741856a1bc486/roles/9baff2b18f2649f7b757ff2e1f63c853 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:40:06.917912 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f72c0a1-1e57-4f09-8e13-93de7414fa67 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:06.929425 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f72c0a1-1e57-4f09-8e13-93de7414fa67 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:07.001414 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bfabd2-eab9-47f5-af4d-e37d5da4e7e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:07.001988 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bfabd2-eab9-47f5-af4d-e37d5da4e7e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:07.002276 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bfabd2-eab9-47f5-af4d-e37d5da4e7e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:07.002556 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bfabd2-eab9-47f5-af4d-e37d5da4e7e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:07.003283 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bfabd2-eab9-47f5-af4d-e37d5da4e7e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:07.004915 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bfabd2-eab9-47f5-af4d-e37d5da4e7e9 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:07.015635 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 947/1872] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:40:06 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 14:40:07.019799 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf69561-0e38-45fe-a3d2-39c3501ac9cc admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:07.030074 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf69561-0e38-45fe-a3d2-39c3501ac9cc admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:07.034606 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fe160d-322e-4e28-95b9-0c7576cffc66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:07.035482 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fe160d-322e-4e28-95b9-0c7576cffc66 admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:07.035762 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fe160d-322e-4e28-95b9-0c7576cffc66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:07.036034 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fe160d-322e-4e28-95b9-0c7576cffc66 admin admin] PATH_INFO: `/v3/projects/4d77f0659785486cbf904c5269901eb6/users/b4a3dcf0c791421796d741856a1bc486/roles/9baff2b18f2649f7b757ff2e1f63c853` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:07.038042 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fe160d-322e-4e28-95b9-0c7576cffc66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d77f0659785486cbf904c5269901eb6, user_id=b4a3dcf0c791421796d741856a1bc486, role_id=9baff2b18f2649f7b757ff2e1f63c853)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:07.040705 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fe160d-322e-4e28-95b9-0c7576cffc66 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:07.050658 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 926/1873] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:40:07 2022] PUT /identity/v3/projects/4d77f0659785486cbf904c5269901eb6/users/b4a3dcf0c791421796d741856a1bc486/roles/9baff2b18f2649f7b757ff2e1f63c853 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:40:07.064444 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dbd5c3-b8fa-4653-a30f-cdddc27ac3a6 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:07.064748 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dbd5c3-b8fa-4653-a30f-cdddc27ac3a6 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:07.065040 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dbd5c3-b8fa-4653-a30f-cdddc27ac3a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:07.094722 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-39dbd5c3-b8fa-4653-a30f-cdddc27ac3a6 None None] MFA Rules not processed for user `b4a3dcf0c791421796d741856a1bc486`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:40:07.094722 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-39dbd5c3-b8fa-4653-a30f-cdddc27ac3a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:07.162500 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 948/1874] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:40:07 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:40:07.188778 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fe160d-322e-4e28-95b9-0c7576cffc66 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:07.190702 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-e9fe160d-322e-4e28-95b9-0c7576cffc66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:07.247810 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fe160d-322e-4e28-95b9-0c7576cffc66 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:07.252174 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbf6222-26fc-4494-bc8a-5a021f074284 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d81f597cb44c56a966537089bcce4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:07.252820 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbf6222-26fc-4494-bc8a-5a021f074284 service cinder] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:07.253111 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbf6222-26fc-4494-bc8a-5a021f074284 service cinder] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:07.253393 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbf6222-26fc-4494-bc8a-5a021f074284 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:07.263020 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbf6222-26fc-4494-bc8a-5a021f074284 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:07.265238 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbf6222-26fc-4494-bc8a-5a021f074284 service cinder] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:07.274703 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 927/1875] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:40:07 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:40:07.302072 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dbd5c3-b8fa-4653-a30f-cdddc27ac3a6 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:07.383828 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dbd5c3-b8fa-4653-a30f-cdddc27ac3a6 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:07.389035 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b1bcef-b240-4852-a6cc-98c453f7262e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:07.389956 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b1bcef-b240-4852-a6cc-98c453f7262e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:07.390246 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b1bcef-b240-4852-a6cc-98c453f7262e admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:07.390531 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b1bcef-b240-4852-a6cc-98c453f7262e admin admin] PATH_INFO: `/v3/users/a0f8ad0379cf48e7b821d12171382613` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:07.391777 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b1bcef-b240-4852-a6cc-98c453f7262e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0f8ad0379cf48e7b821d12171382613)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:07.393868 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b1bcef-b240-4852-a6cc-98c453f7262e admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:07.618835 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-c7b1bcef-b240-4852-a6cc-98c453f7262e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0f8ad0379cf48e7b821d12171382613'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:07.620048 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-c7b1bcef-b240-4852-a6cc-98c453f7262e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0f8ad0379cf48e7b821d12171382613'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:07.621052 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-c7b1bcef-b240-4852-a6cc-98c453f7262e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0f8ad0379cf48e7b821d12171382613'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:07.630877 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-c7b1bcef-b240-4852-a6cc-98c453f7262e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0f8ad0379cf48e7b821d12171382613'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:07.644030 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-c7b1bcef-b240-4852-a6cc-98c453f7262e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0f8ad0379cf48e7b821d12171382613'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:07.751843 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:40:07.757281 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 949/1876] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:40:07 2022] DELETE /identity/v3/users/a0f8ad0379cf48e7b821d12171382613 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:40:07.762569 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761cab36-4a48-4181-833c-b69a4cc82da2 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:07.762862 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761cab36-4a48-4181-833c-b69a4cc82da2 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:07.763142 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761cab36-4a48-4181-833c-b69a4cc82da2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:07.781108 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-761cab36-4a48-4181-833c-b69a4cc82da2 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:40:07.782309 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-761cab36-4a48-4181-833c-b69a4cc82da2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:07.845389 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 928/1877] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:40:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:40:07.856743 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b1bcef-b240-4852-a6cc-98c453f7262e admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:07.858887 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-c7b1bcef-b240-4852-a6cc-98c453f7262e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:07.928245 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b1bcef-b240-4852-a6cc-98c453f7262e admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:07.932187 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35df6a6c-47e9-4343-99e0-2a83bc983ad3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:07.932890 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35df6a6c-47e9-4343-99e0-2a83bc983ad3 service neutron] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:07.933179 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35df6a6c-47e9-4343-99e0-2a83bc983ad3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:07.933461 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35df6a6c-47e9-4343-99e0-2a83bc983ad3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:07.945354 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35df6a6c-47e9-4343-99e0-2a83bc983ad3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:07.946799 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35df6a6c-47e9-4343-99e0-2a83bc983ad3 service neutron] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:07.957827 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 950/1878] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:40:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:40:08.140639 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:40:08.375905 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761cab36-4a48-4181-833c-b69a4cc82da2 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:08.377902 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-761cab36-4a48-4181-833c-b69a4cc82da2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:08.395757 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761cab36-4a48-4181-833c-b69a4cc82da2 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:08.401164 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dbfec2-7bed-44fb-b975-7129121752ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:08.402390 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dbfec2-7bed-44fb-b975-7129121752ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:08.402706 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dbfec2-7bed-44fb-b975-7129121752ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:08.403011 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dbfec2-7bed-44fb-b975-7129121752ce admin admin] PATH_INFO: `/v3/projects/33519627a442485490b84fd2a5c1c8ee` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:08.404247 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dbfec2-7bed-44fb-b975-7129121752ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=33519627a442485490b84fd2a5c1c8ee)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:08.406355 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dbfec2-7bed-44fb-b975-7129121752ce admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:08.507961 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35df6a6c-47e9-4343-99e0-2a83bc983ad3 service neutron] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:08.509997 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-35df6a6c-47e9-4343-99e0-2a83bc983ad3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:08.536907 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35df6a6c-47e9-4343-99e0-2a83bc983ad3 service neutron] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:08.547293 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0ce46c-6cc0-484a-9447-7b9c34019592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:08.548930 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0ce46c-6cc0-484a-9447-7b9c34019592 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:08.549216 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0ce46c-6cc0-484a-9447-7b9c34019592 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:08.549495 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0ce46c-6cc0-484a-9447-7b9c34019592 admin admin] PATH_INFO: `/v3/users/5a2986dc97c14e1bb81fe2c6971baec4` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:08.552232 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0ce46c-6cc0-484a-9447-7b9c34019592 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a2986dc97c14e1bb81fe2c6971baec4)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:08.554806 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0ce46c-6cc0-484a-9447-7b9c34019592 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:08.568824 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-55dbfec2-7bed-44fb-b975-7129121752ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33519627a442485490b84fd2a5c1c8ee', 'request_id': 'req-55dbfec2-7bed-44fb-b975-7129121752ce'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:08.624917 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-55dbfec2-7bed-44fb-b975-7129121752ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33519627a442485490b84fd2a5c1c8ee', 'request_id': 'req-55dbfec2-7bed-44fb-b975-7129121752ce'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:08.633619 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-55dbfec2-7bed-44fb-b975-7129121752ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33519627a442485490b84fd2a5c1c8ee', 'request_id': 'req-55dbfec2-7bed-44fb-b975-7129121752ce'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:08.643650 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-55dbfec2-7bed-44fb-b975-7129121752ce admin admin] The token cache is being invalidate because project 33519627a442485490b84fd2a5c1c8ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:40:08.644157 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-55dbfec2-7bed-44fb-b975-7129121752ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:08.645016 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-55dbfec2-7bed-44fb-b975-7129121752ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:08.647402 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 929/1879] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:40:08 2022] DELETE /identity/v3/projects/33519627a442485490b84fd2a5c1c8ee => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:40:08.649643 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:40:08.653965 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dbfec2-7bed-44fb-b975-7129121752ce admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:08.656358 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-55dbfec2-7bed-44fb-b975-7129121752ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:08.745163 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dbfec2-7bed-44fb-b975-7129121752ce admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:08.751371 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc9d328-7c37-4324-ab4e-d54d1d84347e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:08.752309 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc9d328-7c37-4324-ab4e-d54d1d84347e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:08.752912 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc9d328-7c37-4324-ab4e-d54d1d84347e admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:08.752912 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc9d328-7c37-4324-ab4e-d54d1d84347e admin admin] PATH_INFO: `/v3/users/b4a3dcf0c791421796d741856a1bc486` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:08.754096 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc9d328-7c37-4324-ab4e-d54d1d84347e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4a3dcf0c791421796d741856a1bc486)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:08.756078 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc9d328-7c37-4324-ab4e-d54d1d84347e admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:08.767175 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:40:08.769314 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-ef0ce46c-6cc0-484a-9447-7b9c34019592 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a2986dc97c14e1bb81fe2c6971baec4'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:08.770025 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-ef0ce46c-6cc0-484a-9447-7b9c34019592 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a2986dc97c14e1bb81fe2c6971baec4'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:08.771481 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-ef0ce46c-6cc0-484a-9447-7b9c34019592 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a2986dc97c14e1bb81fe2c6971baec4'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:08.781443 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-ef0ce46c-6cc0-484a-9447-7b9c34019592 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a2986dc97c14e1bb81fe2c6971baec4'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:08.801344 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-ef0ce46c-6cc0-484a-9447-7b9c34019592 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a2986dc97c14e1bb81fe2c6971baec4'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:08.883338 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 951/1880] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:40:08 2022] DELETE /identity/v3/users/5a2986dc97c14e1bb81fe2c6971baec4 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:40:08.889543 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818bd910-2adc-4734-983c-07f9098d43a9 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:08.889824 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818bd910-2adc-4734-983c-07f9098d43a9 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:08.890109 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818bd910-2adc-4734-983c-07f9098d43a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:08.907864 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-818bd910-2adc-4734-983c-07f9098d43a9 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:40:08.909140 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-818bd910-2adc-4734-983c-07f9098d43a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:08.971381 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 952/1881] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:40:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:40:08.981554 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818bd910-2adc-4734-983c-07f9098d43a9 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:08.983044 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-818bd910-2adc-4734-983c-07f9098d43a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:09.031365 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-ffc9d328-7c37-4324-ab4e-d54d1d84347e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4a3dcf0c791421796d741856a1bc486'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:09.031365 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-ffc9d328-7c37-4324-ab4e-d54d1d84347e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4a3dcf0c791421796d741856a1bc486'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:09.031755 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-ffc9d328-7c37-4324-ab4e-d54d1d84347e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4a3dcf0c791421796d741856a1bc486'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:09.039338 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-ffc9d328-7c37-4324-ab4e-d54d1d84347e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4a3dcf0c791421796d741856a1bc486'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:09.053731 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-ffc9d328-7c37-4324-ab4e-d54d1d84347e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4a3dcf0c791421796d741856a1bc486'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:09.066017 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818bd910-2adc-4734-983c-07f9098d43a9 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:09.069957 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9c4d0c-052a-4008-8570-f4f9f92c10d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:09.070714 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9c4d0c-052a-4008-8570-f4f9f92c10d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:09.070999 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9c4d0c-052a-4008-8570-f4f9f92c10d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:09.071277 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9c4d0c-052a-4008-8570-f4f9f92c10d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:09.073437 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-ac9c4d0c-052a-4008-8570-f4f9f92c10d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:09.086795 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9c4d0c-052a-4008-8570-f4f9f92c10d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:09.088235 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9c4d0c-052a-4008-8570-f4f9f92c10d9 service neutron] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:09.099270 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 953/1882] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:40:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:40:09.169833 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 930/1883] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:40:08 2022] DELETE /identity/v3/users/b4a3dcf0c791421796d741856a1bc486 => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:40:09.610148 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9c4d0c-052a-4008-8570-f4f9f92c10d9 service neutron] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:09.611670 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-ac9c4d0c-052a-4008-8570-f4f9f92c10d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:09.624830 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9c4d0c-052a-4008-8570-f4f9f92c10d9 service neutron] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:09.651765 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc9d328-7c37-4324-ab4e-d54d1d84347e admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:09.653270 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-ffc9d328-7c37-4324-ab4e-d54d1d84347e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:09.751293 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15ab98e-6541-4d9a-832d-090807093661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:09.752251 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15ab98e-6541-4d9a-832d-090807093661 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:09.752679 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15ab98e-6541-4d9a-832d-090807093661 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:09.753081 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15ab98e-6541-4d9a-832d-090807093661 admin admin] PATH_INFO: `/v3/projects/e605046e2c214fdf9fc0c821acda2d4e` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:09.757466 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15ab98e-6541-4d9a-832d-090807093661 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e605046e2c214fdf9fc0c821acda2d4e)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:09.757466 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15ab98e-6541-4d9a-832d-090807093661 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:09.763296 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc9d328-7c37-4324-ab4e-d54d1d84347e admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:09.769618 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d87268f-1ef4-46d2-9e37-6dc317948181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:09.770493 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d87268f-1ef4-46d2-9e37-6dc317948181 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:09.770772 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d87268f-1ef4-46d2-9e37-6dc317948181 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:09.772516 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d87268f-1ef4-46d2-9e37-6dc317948181 admin admin] PATH_INFO: `/v3/projects/4d77f0659785486cbf904c5269901eb6` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:09.772516 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d87268f-1ef4-46d2-9e37-6dc317948181 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d77f0659785486cbf904c5269901eb6)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:09.774383 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d87268f-1ef4-46d2-9e37-6dc317948181 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:09.801001 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:40:09.912239 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-e15ab98e-6541-4d9a-832d-090807093661 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e605046e2c214fdf9fc0c821acda2d4e', 'request_id': 'req-e15ab98e-6541-4d9a-832d-090807093661'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:09.965807 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-7d87268f-1ef4-46d2-9e37-6dc317948181 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d77f0659785486cbf904c5269901eb6', 'request_id': 'req-7d87268f-1ef4-46d2-9e37-6dc317948181'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:09.991079 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-e15ab98e-6541-4d9a-832d-090807093661 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e605046e2c214fdf9fc0c821acda2d4e', 'request_id': 'req-e15ab98e-6541-4d9a-832d-090807093661'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:10.005611 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-e15ab98e-6541-4d9a-832d-090807093661 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e605046e2c214fdf9fc0c821acda2d4e', 'request_id': 'req-e15ab98e-6541-4d9a-832d-090807093661'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:10.018658 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-e15ab98e-6541-4d9a-832d-090807093661 admin admin] The token cache is being invalidate because project e605046e2c214fdf9fc0c821acda2d4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:40:10.019186 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-e15ab98e-6541-4d9a-832d-090807093661 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:10.020878 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:40:10.021570 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-e15ab98e-6541-4d9a-832d-090807093661 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:10.023833 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 954/1884] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:40:09 2022] DELETE /identity/v3/projects/e605046e2c214fdf9fc0c821acda2d4e => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:40:10.030314 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15ab98e-6541-4d9a-832d-090807093661 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:10.031760 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-e15ab98e-6541-4d9a-832d-090807093661 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:10.077635 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-7d87268f-1ef4-46d2-9e37-6dc317948181 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d77f0659785486cbf904c5269901eb6', 'request_id': 'req-7d87268f-1ef4-46d2-9e37-6dc317948181'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:10.084622 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-7d87268f-1ef4-46d2-9e37-6dc317948181 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d77f0659785486cbf904c5269901eb6', 'request_id': 'req-7d87268f-1ef4-46d2-9e37-6dc317948181'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:10.099672 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15ab98e-6541-4d9a-832d-090807093661 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:10.100147 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-7d87268f-1ef4-46d2-9e37-6dc317948181 admin admin] The token cache is being invalidate because project 4d77f0659785486cbf904c5269901eb6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:40:10.100485 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-7d87268f-1ef4-46d2-9e37-6dc317948181 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:10.101156 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-7d87268f-1ef4-46d2-9e37-6dc317948181 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:10.102822 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 931/1885] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:40:09 2022] DELETE /identity/v3/projects/4d77f0659785486cbf904c5269901eb6 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:40:10.111113 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c10e72-82ed-455b-b5c6-3c2f9f1636bc None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:10.113210 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c10e72-82ed-455b-b5c6-3c2f9f1636bc None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:10.113210 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c10e72-82ed-455b-b5c6-3c2f9f1636bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:10.135117 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-a3c10e72-82ed-455b-b5c6-3c2f9f1636bc None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:40:10.136622 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-a3c10e72-82ed-455b-b5c6-3c2f9f1636bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:10.180797 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025ecc43-9b22-4a83-9b2f-7a3fcf9fdfdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:10.181726 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025ecc43-9b22-4a83-9b2f-7a3fcf9fdfdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:10.182192 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025ecc43-9b22-4a83-9b2f-7a3fcf9fdfdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:10.182405 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025ecc43-9b22-4a83-9b2f-7a3fcf9fdfdd admin admin] PATH_INFO: `/v3/users/6389fa395ba74b6cb646df091cb1c544` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:10.183721 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025ecc43-9b22-4a83-9b2f-7a3fcf9fdfdd admin admin] RBAC: Authorizing `identity:delete_user(user_id=6389fa395ba74b6cb646df091cb1c544)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:10.186587 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025ecc43-9b22-4a83-9b2f-7a3fcf9fdfdd admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:10.212646 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 932/1886] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:40:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:40:10.212646 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c10e72-82ed-455b-b5c6-3c2f9f1636bc None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:10.216524 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-a3c10e72-82ed-455b-b5c6-3c2f9f1636bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:10.227116 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c10e72-82ed-455b-b5c6-3c2f9f1636bc None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:10.231944 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a335f0a0-7fb1-4392-94df-8250b212e4fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:10.233427 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a335f0a0-7fb1-4392-94df-8250b212e4fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:10.233722 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a335f0a0-7fb1-4392-94df-8250b212e4fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:10.233998 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a335f0a0-7fb1-4392-94df-8250b212e4fe admin admin] PATH_INFO: `/v3/users/80baf17adab84876b85fc67b03ef30f5` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:10.235213 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a335f0a0-7fb1-4392-94df-8250b212e4fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=80baf17adab84876b85fc67b03ef30f5)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:10.237158 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a335f0a0-7fb1-4392-94df-8250b212e4fe admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:10.420929 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-025ecc43-9b22-4a83-9b2f-7a3fcf9fdfdd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6389fa395ba74b6cb646df091cb1c544'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:10.421841 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-025ecc43-9b22-4a83-9b2f-7a3fcf9fdfdd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6389fa395ba74b6cb646df091cb1c544'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:10.422456 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-025ecc43-9b22-4a83-9b2f-7a3fcf9fdfdd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6389fa395ba74b6cb646df091cb1c544'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:10.429445 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-025ecc43-9b22-4a83-9b2f-7a3fcf9fdfdd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6389fa395ba74b6cb646df091cb1c544'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:10.433360 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a335f0a0-7fb1-4392-94df-8250b212e4fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80baf17adab84876b85fc67b03ef30f5'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:10.434019 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a335f0a0-7fb1-4392-94df-8250b212e4fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80baf17adab84876b85fc67b03ef30f5'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:10.434707 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a335f0a0-7fb1-4392-94df-8250b212e4fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80baf17adab84876b85fc67b03ef30f5'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:10.441518 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-025ecc43-9b22-4a83-9b2f-7a3fcf9fdfdd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6389fa395ba74b6cb646df091cb1c544'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:10.441642 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a335f0a0-7fb1-4392-94df-8250b212e4fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80baf17adab84876b85fc67b03ef30f5'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:10.454310 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a335f0a0-7fb1-4392-94df-8250b212e4fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80baf17adab84876b85fc67b03ef30f5'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:10.522981 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 955/1887] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:40:10 2022] DELETE /identity/v3/users/6389fa395ba74b6cb646df091cb1c544 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:40:10.524337 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b581e981-013d-4e45-91ef-1bb026d59316 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:10.524664 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b581e981-013d-4e45-91ef-1bb026d59316 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:10.524950 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b581e981-013d-4e45-91ef-1bb026d59316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:10.543632 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-b581e981-013d-4e45-91ef-1bb026d59316 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:40:10.544904 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-b581e981-013d-4e45-91ef-1bb026d59316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:10.603558 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 933/1888] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:40:10 2022] DELETE /identity/v3/users/80baf17adab84876b85fc67b03ef30f5 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:40:10.608739 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f72b095-ce8d-4328-b3fe-bf6c41ad52b9 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:10.609034 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f72b095-ce8d-4328-b3fe-bf6c41ad52b9 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:10.609320 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f72b095-ce8d-4328-b3fe-bf6c41ad52b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:10.620152 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 956/1889] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:40:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:40:10.621396 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b581e981-013d-4e45-91ef-1bb026d59316 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:10.631439 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b581e981-013d-4e45-91ef-1bb026d59316 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:10.634917 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-1f72b095-ce8d-4328-b3fe-bf6c41ad52b9 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:40:10.636148 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-1f72b095-ce8d-4328-b3fe-bf6c41ad52b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:10.696094 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85615beb-6316-4f81-bf27-a6c692be3758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:10.696722 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85615beb-6316-4f81-bf27-a6c692be3758 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:10.697011 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85615beb-6316-4f81-bf27-a6c692be3758 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:10.697294 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85615beb-6316-4f81-bf27-a6c692be3758 admin admin] PATH_INFO: `/v3/domains` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:10.698026 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85615beb-6316-4f81-bf27-a6c692be3758 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:40:10.698399 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 934/1890] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:40:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:40:10.698509 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85615beb-6316-4f81-bf27-a6c692be3758 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:10.700042 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85615beb-6316-4f81-bf27-a6c692be3758 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:10.710913 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f72b095-ce8d-4328-b3fe-bf6c41ad52b9 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:10.712324 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-1f72b095-ce8d-4328-b3fe-bf6c41ad52b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:10.714387 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 957/1891] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jun 7 14:40:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:40:10.718983 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85615beb-6316-4f81-bf27-a6c692be3758 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:10.740658 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85615beb-6316-4f81-bf27-a6c692be3758 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:10.747327 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d6e095-e7e6-475f-a5f5-d53edb1ade22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:10.751452 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d6e095-e7e6-475f-a5f5-d53edb1ade22 admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:10.751452 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d6e095-e7e6-475f-a5f5-d53edb1ade22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:10.751452 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d6e095-e7e6-475f-a5f5-d53edb1ade22 admin admin] PATH_INFO: `/v3/projects` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:10.751452 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d6e095-e7e6-475f-a5f5-d53edb1ade22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:10.753081 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d6e095-e7e6-475f-a5f5-d53edb1ade22 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:10.812768 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f72b095-ce8d-4328-b3fe-bf6c41ad52b9 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:10.837631 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf374229-4f5b-45c9-92a4-19a6ef2f3722 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:10.840494 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf374229-4f5b-45c9-92a4-19a6ef2f3722 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:10.842525 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf374229-4f5b-45c9-92a4-19a6ef2f3722 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:10.844503 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf374229-4f5b-45c9-92a4-19a6ef2f3722 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:10.866176 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf374229-4f5b-45c9-92a4-19a6ef2f3722 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:10.869625 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf374229-4f5b-45c9-92a4-19a6ef2f3722 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:10.891237 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 935/1892] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:40:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:40:10.976149 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 958/1893] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:40:10 2022] POST /identity/v3/projects => generated 379 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:40:10.993265 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf374229-4f5b-45c9-92a4-19a6ef2f3722 service neutron] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:11.001928 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf374229-4f5b-45c9-92a4-19a6ef2f3722 service neutron] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:11.066020 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:40:11.164712 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eb8755-0fee-4891-96ef-f87a0320b05f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:11.165802 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eb8755-0fee-4891-96ef-f87a0320b05f admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:11.166095 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eb8755-0fee-4891-96ef-f87a0320b05f admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:11.166377 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eb8755-0fee-4891-96ef-f87a0320b05f admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:11.167085 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56eb8755-0fee-4891-96ef-f87a0320b05f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:11.169389 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56eb8755-0fee-4891-96ef-f87a0320b05f admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:11.300775 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:40:11.319170 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 936/1894] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:40:10 2022] POST /identity/v3/users => generated 290 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:40:11.322589 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d6e095-e7e6-475f-a5f5-d53edb1ade22 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:11.325819 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-43d6e095-e7e6-475f-a5f5-d53edb1ade22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:11.328580 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eb8755-0fee-4891-96ef-f87a0320b05f admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:11.348891 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eb8755-0fee-4891-96ef-f87a0320b05f admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:11.350400 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d6e095-e7e6-475f-a5f5-d53edb1ade22 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:11.357240 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0c10d4-4f83-40c1-a377-6f2910361e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:11.359073 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c0f61b-beec-4c93-a53e-c96d125e0d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:11.359606 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0c10d4-4f83-40c1-a377-6f2910361e43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:11.360082 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0c10d4-4f83-40c1-a377-6f2910361e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:11.361849 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c0f61b-beec-4c93-a53e-c96d125e0d1c admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:11.361849 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c0f61b-beec-4c93-a53e-c96d125e0d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:11.361849 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c0f61b-beec-4c93-a53e-c96d125e0d1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:11.362574 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0c10d4-4f83-40c1-a377-6f2910361e43 admin admin] PATH_INFO: `/v3/projects/9218efbcd2824c769e2101488c9f3c57` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:11.362905 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c0f61b-beec-4c93-a53e-c96d125e0d1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:11.364633 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0c10d4-4f83-40c1-a377-6f2910361e43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9218efbcd2824c769e2101488c9f3c57)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:11.364794 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c0f61b-beec-4c93-a53e-c96d125e0d1c admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:11.367103 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0c10d4-4f83-40c1-a377-6f2910361e43 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:11.379121 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 937/1895] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:40:11 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:40:11.383507 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c0f61b-beec-4c93-a53e-c96d125e0d1c admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:11.401205 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c0f61b-beec-4c93-a53e-c96d125e0d1c admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:11.408960 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333cc019-16af-4e3c-a032-e9bd46d28ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:11.409787 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333cc019-16af-4e3c-a032-e9bd46d28ce1 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:11.410173 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333cc019-16af-4e3c-a032-e9bd46d28ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:11.410625 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333cc019-16af-4e3c-a032-e9bd46d28ce1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:11.411471 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333cc019-16af-4e3c-a032-e9bd46d28ce1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:11.414211 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333cc019-16af-4e3c-a032-e9bd46d28ce1 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:11.429103 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 938/1896] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:40:11 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 14:40:11.431047 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333cc019-16af-4e3c-a032-e9bd46d28ce1 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:11.442461 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333cc019-16af-4e3c-a032-e9bd46d28ce1 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:11.450340 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b05742-440f-42c8-a745-b4844515088a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:11.451419 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b05742-440f-42c8-a745-b4844515088a admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:11.452407 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b05742-440f-42c8-a745-b4844515088a admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:11.452792 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b05742-440f-42c8-a745-b4844515088a admin admin] PATH_INFO: `/v3/projects/735825f6725d409e9aa7e442497a19bb/users/1eac7fccb72b4328bd339929472abe0e/roles/3771341b3be641139e752aab690397a4` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:11.471080 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b05742-440f-42c8-a745-b4844515088a admin admin] RBAC: Authorizing `identity:create_grant(project_id=735825f6725d409e9aa7e442497a19bb, user_id=1eac7fccb72b4328bd339929472abe0e, role_id=3771341b3be641139e752aab690397a4)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:11.474609 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b05742-440f-42c8-a745-b4844515088a admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:11.588337 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 939/1897] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:40:11 2022] PUT /identity/v3/projects/735825f6725d409e9aa7e442497a19bb/users/1eac7fccb72b4328bd339929472abe0e/roles/3771341b3be641139e752aab690397a4 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:40:11.590227 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af7defa-e454-433f-a9f7-94c8660370c5 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:11.590524 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af7defa-e454-433f-a9f7-94c8660370c5 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:11.590906 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af7defa-e454-433f-a9f7-94c8660370c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:11.593993 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-ae0c10d4-4f83-40c1-a377-6f2910361e43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9218efbcd2824c769e2101488c9f3c57', 'request_id': 'req-ae0c10d4-4f83-40c1-a377-6f2910361e43'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:11.642257 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-1af7defa-e454-433f-a9f7-94c8660370c5 None None] MFA Rules not processed for user `1eac7fccb72b4328bd339929472abe0e`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:40:11.643511 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-1af7defa-e454-433f-a9f7-94c8660370c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:11.711731 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-ae0c10d4-4f83-40c1-a377-6f2910361e43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9218efbcd2824c769e2101488c9f3c57', 'request_id': 'req-ae0c10d4-4f83-40c1-a377-6f2910361e43'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:11.721597 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-ae0c10d4-4f83-40c1-a377-6f2910361e43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9218efbcd2824c769e2101488c9f3c57', 'request_id': 'req-ae0c10d4-4f83-40c1-a377-6f2910361e43'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:11.748198 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-ae0c10d4-4f83-40c1-a377-6f2910361e43 admin admin] The token cache is being invalidate because project 9218efbcd2824c769e2101488c9f3c57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:40:11.749589 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-ae0c10d4-4f83-40c1-a377-6f2910361e43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:11.752945 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-ae0c10d4-4f83-40c1-a377-6f2910361e43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:11.755193 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 959/1898] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:40:11 2022] DELETE /identity/v3/projects/9218efbcd2824c769e2101488c9f3c57 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:40:11.769793 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0c10d4-4f83-40c1-a377-6f2910361e43 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:11.777814 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-ae0c10d4-4f83-40c1-a377-6f2910361e43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:11.783508 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 940/1899] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:40:11 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:40:11.794671 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af7defa-e454-433f-a9f7-94c8660370c5 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:11.797787 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-1af7defa-e454-433f-a9f7-94c8660370c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:11.912247 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af7defa-e454-433f-a9f7-94c8660370c5 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:11.913804 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0149b756-68e9-412a-8e2e-35c6950977ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:11.914772 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0149b756-68e9-412a-8e2e-35c6950977ac admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:11.915169 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0149b756-68e9-412a-8e2e-35c6950977ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:11.915665 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0149b756-68e9-412a-8e2e-35c6950977ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:11.916556 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0149b756-68e9-412a-8e2e-35c6950977ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:11.929648 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0149b756-68e9-412a-8e2e-35c6950977ac admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:11.988989 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0c10d4-4f83-40c1-a377-6f2910361e43 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:11.995561 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9fb83f-9f5e-4c47-90cb-7ff977f0281b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:11.996734 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9fb83f-9f5e-4c47-90cb-7ff977f0281b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:12.001994 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9fb83f-9f5e-4c47-90cb-7ff977f0281b admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:12.001994 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9fb83f-9f5e-4c47-90cb-7ff977f0281b admin admin] PATH_INFO: `/v3/users/330bfaa1f3924f6fa0c4f27c28e1834e` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:12.001994 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9fb83f-9f5e-4c47-90cb-7ff977f0281b admin admin] RBAC: Authorizing `identity:delete_user(user_id=330bfaa1f3924f6fa0c4f27c28e1834e)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:12.002414 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9fb83f-9f5e-4c47-90cb-7ff977f0281b admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:12.009949 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 941/1900] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:40:11 2022] POST /identity/v3/projects => generated 377 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:40:12.011205 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0149b756-68e9-412a-8e2e-35c6950977ac admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:12.016087 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-0149b756-68e9-412a-8e2e-35c6950977ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:12.118383 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0149b756-68e9-412a-8e2e-35c6950977ac admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:12.123182 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f11347-6346-49cd-a3aa-521fe5fccf53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:12.124796 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f11347-6346-49cd-a3aa-521fe5fccf53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:12.125100 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f11347-6346-49cd-a3aa-521fe5fccf53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:12.125362 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f11347-6346-49cd-a3aa-521fe5fccf53 admin admin] PATH_INFO: `/v3/projects/9b7a7c200543413c930cab774ae2c38f` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:12.126643 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f11347-6346-49cd-a3aa-521fe5fccf53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b7a7c200543413c930cab774ae2c38f)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:12.128787 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f11347-6346-49cd-a3aa-521fe5fccf53 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:12.185684 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-3c9fb83f-9f5e-4c47-90cb-7ff977f0281b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '330bfaa1f3924f6fa0c4f27c28e1834e'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:12.187016 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-3c9fb83f-9f5e-4c47-90cb-7ff977f0281b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '330bfaa1f3924f6fa0c4f27c28e1834e'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:12.187659 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-3c9fb83f-9f5e-4c47-90cb-7ff977f0281b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '330bfaa1f3924f6fa0c4f27c28e1834e'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:12.195697 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-3c9fb83f-9f5e-4c47-90cb-7ff977f0281b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '330bfaa1f3924f6fa0c4f27c28e1834e'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:12.214578 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-3c9fb83f-9f5e-4c47-90cb-7ff977f0281b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '330bfaa1f3924f6fa0c4f27c28e1834e'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:12.258054 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-90f11347-6346-49cd-a3aa-521fe5fccf53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b7a7c200543413c930cab774ae2c38f', 'request_id': 'req-90f11347-6346-49cd-a3aa-521fe5fccf53'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:12.320066 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 960/1901] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:40:11 2022] DELETE /identity/v3/users/330bfaa1f3924f6fa0c4f27c28e1834e => generated 0 bytes in 561 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:40:12.321510 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9fb83f-9f5e-4c47-90cb-7ff977f0281b admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:12.322997 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-3c9fb83f-9f5e-4c47-90cb-7ff977f0281b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:12.375675 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-90f11347-6346-49cd-a3aa-521fe5fccf53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b7a7c200543413c930cab774ae2c38f', 'request_id': 'req-90f11347-6346-49cd-a3aa-521fe5fccf53'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:12.386901 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-90f11347-6346-49cd-a3aa-521fe5fccf53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b7a7c200543413c930cab774ae2c38f', 'request_id': 'req-90f11347-6346-49cd-a3aa-521fe5fccf53'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:12.403667 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-90f11347-6346-49cd-a3aa-521fe5fccf53 admin admin] The token cache is being invalidate because project 9b7a7c200543413c930cab774ae2c38f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:40:12.404091 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-90f11347-6346-49cd-a3aa-521fe5fccf53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:12.405836 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-90f11347-6346-49cd-a3aa-521fe5fccf53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:12.407930 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 942/1902] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:40:12 2022] DELETE /identity/v3/projects/9b7a7c200543413c930cab774ae2c38f => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:40:12.411953 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f11347-6346-49cd-a3aa-521fe5fccf53 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:12.418384 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9fb83f-9f5e-4c47-90cb-7ff977f0281b admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:12.421596 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-90f11347-6346-49cd-a3aa-521fe5fccf53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:12.425120 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44584d67-ba84-42b9-a090-3387b5b181ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:12.426241 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44584d67-ba84-42b9-a090-3387b5b181ca admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:12.426676 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44584d67-ba84-42b9-a090-3387b5b181ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:12.427257 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44584d67-ba84-42b9-a090-3387b5b181ca admin admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:12.428106 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44584d67-ba84-42b9-a090-3387b5b181ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:12.431236 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44584d67-ba84-42b9-a090-3387b5b181ca admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:12.499605 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f11347-6346-49cd-a3aa-521fe5fccf53 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:12.504496 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59568fbf-59d8-4124-91fb-57ffc8685dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:12.505391 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59568fbf-59d8-4124-91fb-57ffc8685dce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:12.506400 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59568fbf-59d8-4124-91fb-57ffc8685dce admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:12.506400 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59568fbf-59d8-4124-91fb-57ffc8685dce admin admin] PATH_INFO: `/v3/users/0695c1cbf77f4162a0feccc9d83495fe` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:12.507190 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 961/1903] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:40:12 2022] POST /identity/v3/users => generated 289 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:40:12.507357 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59568fbf-59d8-4124-91fb-57ffc8685dce admin admin] RBAC: Authorizing `identity:delete_user(user_id=0695c1cbf77f4162a0feccc9d83495fe)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:12.509199 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59568fbf-59d8-4124-91fb-57ffc8685dce admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:12.511415 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44584d67-ba84-42b9-a090-3387b5b181ca admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:12.513943 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-44584d67-ba84-42b9-a090-3387b5b181ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:12.528320 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44584d67-ba84-42b9-a090-3387b5b181ca admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:12.534045 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90221d38-95f3-4265-909d-eb0e6faf6392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:12.534699 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90221d38-95f3-4265-909d-eb0e6faf6392 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:12.537385 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90221d38-95f3-4265-909d-eb0e6faf6392 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:12.537385 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90221d38-95f3-4265-909d-eb0e6faf6392 admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:12.537385 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90221d38-95f3-4265-909d-eb0e6faf6392 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:12.537786 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90221d38-95f3-4265-909d-eb0e6faf6392 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:12.549176 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 962/1904] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:40:12 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 14:40:12.552988 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90221d38-95f3-4265-909d-eb0e6faf6392 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:12.563150 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90221d38-95f3-4265-909d-eb0e6faf6392 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:12.568222 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2d4198-6681-4be1-9ecf-4326e9088590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:12.569028 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2d4198-6681-4be1-9ecf-4326e9088590 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:12.569427 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2d4198-6681-4be1-9ecf-4326e9088590 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:12.569709 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2d4198-6681-4be1-9ecf-4326e9088590 admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:12.570568 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2d4198-6681-4be1-9ecf-4326e9088590 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:12.572036 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2d4198-6681-4be1-9ecf-4326e9088590 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:12.583428 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 963/1905] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:40:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 14:40:12.587723 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2d4198-6681-4be1-9ecf-4326e9088590 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:12.599026 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2d4198-6681-4be1-9ecf-4326e9088590 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:12.609466 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c621b90-0ceb-4875-a799-bd4e2c5982f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:12.609466 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c621b90-0ceb-4875-a799-bd4e2c5982f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:12.609466 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c621b90-0ceb-4875-a799-bd4e2c5982f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:12.609466 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c621b90-0ceb-4875-a799-bd4e2c5982f9 admin admin] PATH_INFO: `/v3/projects/6bfb0c1a212640c096e4c08cf830464e/users/d95a768ee823408dad5d9c831201a5fc/roles/3771341b3be641139e752aab690397a4` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:12.626412 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c621b90-0ceb-4875-a799-bd4e2c5982f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bfb0c1a212640c096e4c08cf830464e, user_id=d95a768ee823408dad5d9c831201a5fc, role_id=3771341b3be641139e752aab690397a4)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:12.629568 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c621b90-0ceb-4875-a799-bd4e2c5982f9 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:12.683190 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:40:12.684066 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-59568fbf-59d8-4124-91fb-57ffc8685dce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0695c1cbf77f4162a0feccc9d83495fe'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:12.685216 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-59568fbf-59d8-4124-91fb-57ffc8685dce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0695c1cbf77f4162a0feccc9d83495fe'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:12.687199 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-59568fbf-59d8-4124-91fb-57ffc8685dce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0695c1cbf77f4162a0feccc9d83495fe'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:12.695221 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 964/1906] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:40:12 2022] PUT /identity/v3/projects/6bfb0c1a212640c096e4c08cf830464e/users/d95a768ee823408dad5d9c831201a5fc/roles/3771341b3be641139e752aab690397a4 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:40:12.695805 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-59568fbf-59d8-4124-91fb-57ffc8685dce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0695c1cbf77f4162a0feccc9d83495fe'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:12.707325 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefa87e0-d916-4429-997f-f4ebdd3817fa None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:12.708258 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefa87e0-d916-4429-997f-f4ebdd3817fa None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:12.708258 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefa87e0-d916-4429-997f-f4ebdd3817fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:12.712134 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-59568fbf-59d8-4124-91fb-57ffc8685dce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0695c1cbf77f4162a0feccc9d83495fe'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:12.723618 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:40:12.745471 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-fefa87e0-d916-4429-997f-f4ebdd3817fa None None] MFA Rules not processed for user `d95a768ee823408dad5d9c831201a5fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:40:12.746760 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-fefa87e0-d916-4429-997f-f4ebdd3817fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:12.760454 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 943/1907] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:40:12 2022] DELETE /identity/v3/users/0695c1cbf77f4162a0feccc9d83495fe => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:40:12.840096 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 965/1908] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:40:12 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:40:12.840249 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59568fbf-59d8-4124-91fb-57ffc8685dce admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:12.845041 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-59568fbf-59d8-4124-91fb-57ffc8685dce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:12.849216 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefa87e0-d916-4429-997f-f4ebdd3817fa None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:12.850676 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-fefa87e0-d916-4429-997f-f4ebdd3817fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:12.934012 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59568fbf-59d8-4124-91fb-57ffc8685dce admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:12.940984 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ae919f-dcbe-4f4a-9795-2694bb102899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:12.941984 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ae919f-dcbe-4f4a-9795-2694bb102899 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:12.942495 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ae919f-dcbe-4f4a-9795-2694bb102899 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:12.942841 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ae919f-dcbe-4f4a-9795-2694bb102899 admin admin] PATH_INFO: `/v3/projects/9c68a04094a945f59ec7fbbd43ffe086` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:12.942998 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefa87e0-d916-4429-997f-f4ebdd3817fa None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:12.944151 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ae919f-dcbe-4f4a-9795-2694bb102899 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c68a04094a945f59ec7fbbd43ffe086)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:12.946359 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ae919f-dcbe-4f4a-9795-2694bb102899 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:12.958230 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584cf730-b5c4-403d-bc1d-d11df177420d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:12.960649 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584cf730-b5c4-403d-bc1d-d11df177420d admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:12.966130 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584cf730-b5c4-403d-bc1d-d11df177420d admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:12.966130 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584cf730-b5c4-403d-bc1d-d11df177420d admin admin] PATH_INFO: `/v3/projects` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:12.966130 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584cf730-b5c4-403d-bc1d-d11df177420d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:12.966130 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584cf730-b5c4-403d-bc1d-d11df177420d admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:13.098662 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 966/1909] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:40:12 2022] POST /identity/v3/projects => generated 377 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:40:13.099512 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-c6ae919f-dcbe-4f4a-9795-2694bb102899 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c68a04094a945f59ec7fbbd43ffe086', 'request_id': 'req-c6ae919f-dcbe-4f4a-9795-2694bb102899'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:13.103841 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584cf730-b5c4-403d-bc1d-d11df177420d admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:13.186910 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-c6ae919f-dcbe-4f4a-9795-2694bb102899 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c68a04094a945f59ec7fbbd43ffe086', 'request_id': 'req-c6ae919f-dcbe-4f4a-9795-2694bb102899'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:13.195846 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-c6ae919f-dcbe-4f4a-9795-2694bb102899 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c68a04094a945f59ec7fbbd43ffe086', 'request_id': 'req-c6ae919f-dcbe-4f4a-9795-2694bb102899'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:13.208668 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-c6ae919f-dcbe-4f4a-9795-2694bb102899 admin admin] The token cache is being invalidate because project 9c68a04094a945f59ec7fbbd43ffe086 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:40:13.209435 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-c6ae919f-dcbe-4f4a-9795-2694bb102899 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:13.210853 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-c6ae919f-dcbe-4f4a-9795-2694bb102899 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:13.217626 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 944/1910] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:40:12 2022] DELETE /identity/v3/projects/9c68a04094a945f59ec7fbbd43ffe086 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:40:13.217816 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584cf730-b5c4-403d-bc1d-d11df177420d admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:13.219516 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b171b914-1aa6-47b7-8869-71e9a9e078d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:13.219972 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a521968-245d-43ee-9230-3a2c2d8fef8b None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:13.220199 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b171b914-1aa6-47b7-8869-71e9a9e078d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:13.220520 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a521968-245d-43ee-9230-3a2c2d8fef8b None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:13.220649 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b171b914-1aa6-47b7-8869-71e9a9e078d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:13.220803 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b171b914-1aa6-47b7-8869-71e9a9e078d8 admin admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:13.221062 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a521968-245d-43ee-9230-3a2c2d8fef8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:13.221513 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b171b914-1aa6-47b7-8869-71e9a9e078d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:13.223463 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b171b914-1aa6-47b7-8869-71e9a9e078d8 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:13.243321 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-1a521968-245d-43ee-9230-3a2c2d8fef8b None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:40:13.244837 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-1a521968-245d-43ee-9230-3a2c2d8fef8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:13.256425 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 945/1911] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:40:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:40:13.266089 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a7985f-b7dd-4d42-b8bc-b08019c3c789 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:13.266451 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a7985f-b7dd-4d42-b8bc-b08019c3c789 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:13.266850 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a7985f-b7dd-4d42-b8bc-b08019c3c789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:13.288998 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-14a7985f-b7dd-4d42-b8bc-b08019c3c789 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:40:13.290468 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-14a7985f-b7dd-4d42-b8bc-b08019c3c789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:13.308751 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 946/1912] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:40:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:40:13.318552 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a7985f-b7dd-4d42-b8bc-b08019c3c789 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:13.319141 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 967/1913] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:40:13 2022] POST /identity/v3/users => generated 295 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:40:13.328329 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b171b914-1aa6-47b7-8869-71e9a9e078d8 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:13.331870 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-b171b914-1aa6-47b7-8869-71e9a9e078d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:13.337548 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a7985f-b7dd-4d42-b8bc-b08019c3c789 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:13.343916 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2aad9b-a32a-4659-9b58-daa9dd6c5734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:13.344867 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2aad9b-a32a-4659-9b58-daa9dd6c5734 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:13.345244 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2aad9b-a32a-4659-9b58-daa9dd6c5734 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:13.345620 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2aad9b-a32a-4659-9b58-daa9dd6c5734 admin admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:13.346941 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2aad9b-a32a-4659-9b58-daa9dd6c5734 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:40:13.347351 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2aad9b-a32a-4659-9b58-daa9dd6c5734 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:13.348539 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b171b914-1aa6-47b7-8869-71e9a9e078d8 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:13.351893 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2aad9b-a32a-4659-9b58-daa9dd6c5734 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:13.355778 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d4106b-72f1-4a88-9041-fb14aea33d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:13.356777 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d4106b-72f1-4a88-9041-fb14aea33d4a admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:13.357319 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d4106b-72f1-4a88-9041-fb14aea33d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:13.358343 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d4106b-72f1-4a88-9041-fb14aea33d4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:13.359422 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d4106b-72f1-4a88-9041-fb14aea33d4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:13.361717 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d4106b-72f1-4a88-9041-fb14aea33d4a admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:13.376835 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 968/1914] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:40:13 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:40:13.381114 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d4106b-72f1-4a88-9041-fb14aea33d4a admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:13.396770 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d4106b-72f1-4a88-9041-fb14aea33d4a admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:13.403685 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d6bc71-eafb-4809-be61-a22765f2f326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:13.405515 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d6bc71-eafb-4809-be61-a22765f2f326 admin admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:13.405883 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d6bc71-eafb-4809-be61-a22765f2f326 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:13.406172 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d6bc71-eafb-4809-be61-a22765f2f326 admin admin] PATH_INFO: `/v3/projects/08dd2d916fab436484b50a2912f0e946/users/ef6bb1ac3ec04e44963d12ad523a70b9/roles/9baff2b18f2649f7b757ff2e1f63c853` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:13.417439 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 947/1915] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jun 7 14:40:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:40:13.420959 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2aad9b-a32a-4659-9b58-daa9dd6c5734 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:13.435126 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d6bc71-eafb-4809-be61-a22765f2f326 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08dd2d916fab436484b50a2912f0e946, user_id=ef6bb1ac3ec04e44963d12ad523a70b9, role_id=9baff2b18f2649f7b757ff2e1f63c853)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:13.438209 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d6bc71-eafb-4809-be61-a22765f2f326 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:13.461738 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2aad9b-a32a-4659-9b58-daa9dd6c5734 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:13.482097 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47950aaa-1cfb-4309-a02c-ab64eb73c0cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:13.482861 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47950aaa-1cfb-4309-a02c-ab64eb73c0cc admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:13.483137 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47950aaa-1cfb-4309-a02c-ab64eb73c0cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:13.483430 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47950aaa-1cfb-4309-a02c-ab64eb73c0cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:13.484274 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47950aaa-1cfb-4309-a02c-ab64eb73c0cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:13.486094 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47950aaa-1cfb-4309-a02c-ab64eb73c0cc admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:13.549894 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 969/1916] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:40:13 2022] PUT /identity/v3/projects/08dd2d916fab436484b50a2912f0e946/users/ef6bb1ac3ec04e44963d12ad523a70b9/roles/9baff2b18f2649f7b757ff2e1f63c853 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:40:13.550782 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d6bc71-eafb-4809-be61-a22765f2f326 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:13.552843 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-b6d6bc71-eafb-4809-be61-a22765f2f326 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:13.603565 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:40:13.611517 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 948/1917] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:40:13 2022] POST /identity/v3/projects => generated 401 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:40:13.615362 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47950aaa-1cfb-4309-a02c-ab64eb73c0cc admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:13.653387 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d6bc71-eafb-4809-be61-a22765f2f326 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:13.712470 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47950aaa-1cfb-4309-a02c-ab64eb73c0cc admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:13.720148 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87c2392-f1b6-41bd-b86d-4ef4fec056e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:13.721502 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87c2392-f1b6-41bd-b86d-4ef4fec056e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:13.721991 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87c2392-f1b6-41bd-b86d-4ef4fec056e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:13.722454 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87c2392-f1b6-41bd-b86d-4ef4fec056e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:13.723711 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87c2392-f1b6-41bd-b86d-4ef4fec056e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:13.726270 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87c2392-f1b6-41bd-b86d-4ef4fec056e7 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:13.744882 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 949/1918] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:40:13 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:40:13.746404 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87c2392-f1b6-41bd-b86d-4ef4fec056e7 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:13.754132 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118cef46-c86a-424e-aa17-8e9e680dd4d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:13.755640 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118cef46-c86a-424e-aa17-8e9e680dd4d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:13.756035 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118cef46-c86a-424e-aa17-8e9e680dd4d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:13.756479 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118cef46-c86a-424e-aa17-8e9e680dd4d4 admin admin] PATH_INFO: `/v3/projects/9d002e9a174d49b08851bc0eeaf7f81e` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:13.758171 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118cef46-c86a-424e-aa17-8e9e680dd4d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d002e9a174d49b08851bc0eeaf7f81e)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:13.758965 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87c2392-f1b6-41bd-b86d-4ef4fec056e7 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:13.762656 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118cef46-c86a-424e-aa17-8e9e680dd4d4 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:13.765315 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d584bd36-336d-4745-8481-96beac79f038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:13.765974 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d584bd36-336d-4745-8481-96beac79f038 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:13.766255 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d584bd36-336d-4745-8481-96beac79f038 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:13.767618 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d584bd36-336d-4745-8481-96beac79f038 admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:13.767618 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d584bd36-336d-4745-8481-96beac79f038 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:13.769148 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d584bd36-336d-4745-8481-96beac79f038 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:13.880656 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 950/1919] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:40:13 2022] POST /identity/v3/users => generated 301 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:40:13.880656 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d584bd36-336d-4745-8481-96beac79f038 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:13.909902 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d584bd36-336d-4745-8481-96beac79f038 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:13.914575 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-118cef46-c86a-424e-aa17-8e9e680dd4d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d002e9a174d49b08851bc0eeaf7f81e', 'request_id': 'req-118cef46-c86a-424e-aa17-8e9e680dd4d4'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:13.980259 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-118cef46-c86a-424e-aa17-8e9e680dd4d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d002e9a174d49b08851bc0eeaf7f81e', 'request_id': 'req-118cef46-c86a-424e-aa17-8e9e680dd4d4'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:13.988334 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-118cef46-c86a-424e-aa17-8e9e680dd4d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d002e9a174d49b08851bc0eeaf7f81e', 'request_id': 'req-118cef46-c86a-424e-aa17-8e9e680dd4d4'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:13.991996 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea607b3-39bf-4bd0-9642-f92f7c0b471f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:13.993635 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea607b3-39bf-4bd0-9642-f92f7c0b471f admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:13.994067 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea607b3-39bf-4bd0-9642-f92f7c0b471f admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:13.994550 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea607b3-39bf-4bd0-9642-f92f7c0b471f admin admin] PATH_INFO: `/v3/projects/08dd2d916fab436484b50a2912f0e946/users/ef6bb1ac3ec04e44963d12ad523a70b9/roles/9baff2b18f2649f7b757ff2e1f63c853` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:13.997015 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea607b3-39bf-4bd0-9642-f92f7c0b471f admin admin] RBAC: Authorizing `identity:create_grant(project_id=08dd2d916fab436484b50a2912f0e946, user_id=ef6bb1ac3ec04e44963d12ad523a70b9, role_id=9baff2b18f2649f7b757ff2e1f63c853)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:14.000480 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea607b3-39bf-4bd0-9642-f92f7c0b471f admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:14.004001 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-118cef46-c86a-424e-aa17-8e9e680dd4d4 admin admin] The token cache is being invalidate because project 9d002e9a174d49b08851bc0eeaf7f81e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:40:14.004675 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-118cef46-c86a-424e-aa17-8e9e680dd4d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:14.008774 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-118cef46-c86a-424e-aa17-8e9e680dd4d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:14.011004 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 970/1920] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:40:13 2022] DELETE /identity/v3/projects/9d002e9a174d49b08851bc0eeaf7f81e => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:40:14.012009 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118cef46-c86a-424e-aa17-8e9e680dd4d4 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:14.015273 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=84,keystone:INSERT=2,keystone:DELETE=14 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:40:14.018556 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 951/1921] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:40:13 2022] PUT /identity/v3/projects/08dd2d916fab436484b50a2912f0e946/users/ef6bb1ac3ec04e44963d12ad523a70b9/roles/9baff2b18f2649f7b757ff2e1f63c853 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:40:14.021335 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-118cef46-c86a-424e-aa17-8e9e680dd4d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:14.035004 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc13a819-bd94-40d2-a089-1033a3baaadf None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:14.036221 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc13a819-bd94-40d2-a089-1033a3baaadf None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:14.036720 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc13a819-bd94-40d2-a089-1033a3baaadf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:14.044063 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118cef46-c86a-424e-aa17-8e9e680dd4d4 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:14.074217 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-cc13a819-bd94-40d2-a089-1033a3baaadf None None] MFA Rules not processed for user `ef6bb1ac3ec04e44963d12ad523a70b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:40:14.075874 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-cc13a819-bd94-40d2-a089-1033a3baaadf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:14.105970 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf82aebc-3e3f-44ec-94e1-3a6f85ce4d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:14.107040 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf82aebc-3e3f-44ec-94e1-3a6f85ce4d26 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:14.107528 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf82aebc-3e3f-44ec-94e1-3a6f85ce4d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:14.108592 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf82aebc-3e3f-44ec-94e1-3a6f85ce4d26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:14.109866 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf82aebc-3e3f-44ec-94e1-3a6f85ce4d26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:14.112016 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf82aebc-3e3f-44ec-94e1-3a6f85ce4d26 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:14.132604 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 971/1922] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:40:14 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:40:14.132604 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a01da82-58c4-4588-a08c-e7b8c0098372 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:14.132604 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a01da82-58c4-4588-a08c-e7b8c0098372 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:14.132604 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a01da82-58c4-4588-a08c-e7b8c0098372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:14.168534 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-1a01da82-58c4-4588-a08c-e7b8c0098372 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:40:14.170112 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-1a01da82-58c4-4588-a08c-e7b8c0098372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:14.172983 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 952/1923] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:40:14 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:40:14.174419 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc13a819-bd94-40d2-a089-1033a3baaadf None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:14.187181 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 972/1924] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:40:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:40:14.202609 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a01da82-58c4-4588-a08c-e7b8c0098372 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:14.202609 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-1a01da82-58c4-4588-a08c-e7b8c0098372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:14.202849 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc13a819-bd94-40d2-a089-1033a3baaadf None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:14.202849 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec05eb1b-20de-480e-ada5-5795cb46951a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:14.202849 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec05eb1b-20de-480e-ada5-5795cb46951a admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:14.202849 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec05eb1b-20de-480e-ada5-5795cb46951a admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:14.203158 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec05eb1b-20de-480e-ada5-5795cb46951a admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:14.203158 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec05eb1b-20de-480e-ada5-5795cb46951a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:14.203984 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec05eb1b-20de-480e-ada5-5795cb46951a admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:14.216422 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 953/1925] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:40:14 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:40:14.216624 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9525b2-3081-4f4f-ad27-916fb0915706 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:14.216829 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9525b2-3081-4f4f-ad27-916fb0915706 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:14.224126 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9525b2-3081-4f4f-ad27-916fb0915706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:14.238574 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-be9525b2-3081-4f4f-ad27-916fb0915706 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:40:14.239782 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-be9525b2-3081-4f4f-ad27-916fb0915706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:14.250846 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 954/1926] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:40:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:40:14.251652 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9525b2-3081-4f4f-ad27-916fb0915706 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:14.261781 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9525b2-3081-4f4f-ad27-916fb0915706 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:14.267088 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86b5d21-d0ab-4e88-bd53-6abee584d18c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:14.268209 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86b5d21-d0ab-4e88-bd53-6abee584d18c admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:14.268792 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86b5d21-d0ab-4e88-bd53-6abee584d18c admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:14.269222 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86b5d21-d0ab-4e88-bd53-6abee584d18c admin admin] PATH_INFO: `/v3/projects/6bed7945f0ae41a6a2330a1946ad5be9/users/a72376ecb8dc425e821b06b24bd6b9e3/roles/3771341b3be641139e752aab690397a4` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:14.274722 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a01da82-58c4-4588-a08c-e7b8c0098372 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:14.279770 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6907f7ec-2e45-4314-8d8d-acdbffcf55b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d81f597cb44c56a966537089bcce4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:14.280612 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6907f7ec-2e45-4314-8d8d-acdbffcf55b2 service cinder] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:14.280900 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6907f7ec-2e45-4314-8d8d-acdbffcf55b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:14.281821 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6907f7ec-2e45-4314-8d8d-acdbffcf55b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:14.286195 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86b5d21-d0ab-4e88-bd53-6abee584d18c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bed7945f0ae41a6a2330a1946ad5be9, user_id=a72376ecb8dc425e821b06b24bd6b9e3, role_id=3771341b3be641139e752aab690397a4)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:14.289179 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86b5d21-d0ab-4e88-bd53-6abee584d18c admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:14.296409 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6907f7ec-2e45-4314-8d8d-acdbffcf55b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:14.297910 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6907f7ec-2e45-4314-8d8d-acdbffcf55b2 service cinder] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:14.308607 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 973/1927] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:40:14 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:40:14.309448 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6907f7ec-2e45-4314-8d8d-acdbffcf55b2 service cinder] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:14.324848 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6907f7ec-2e45-4314-8d8d-acdbffcf55b2 service cinder] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:14.327854 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebee0c7-3c43-4335-8dc8-0f4474766a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:14.328559 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebee0c7-3c43-4335-8dc8-0f4474766a60 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:14.329083 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebee0c7-3c43-4335-8dc8-0f4474766a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:14.329185 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebee0c7-3c43-4335-8dc8-0f4474766a60 admin admin] PATH_INFO: `/v3/domains` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:14.329943 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebee0c7-3c43-4335-8dc8-0f4474766a60 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:40:14.330257 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebee0c7-3c43-4335-8dc8-0f4474766a60 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:14.331863 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebee0c7-3c43-4335-8dc8-0f4474766a60 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:14.345133 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 974/1928] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jun 7 14:40:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:40:14.351026 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebee0c7-3c43-4335-8dc8-0f4474766a60 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:14.361669 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebee0c7-3c43-4335-8dc8-0f4474766a60 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:14.368204 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 955/1929] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:40:14 2022] PUT /identity/v3/projects/6bed7945f0ae41a6a2330a1946ad5be9/users/a72376ecb8dc425e821b06b24bd6b9e3/roles/3771341b3be641139e752aab690397a4 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:40:14.381631 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b94eb6-26c4-48d5-9112-54fcb9dbe9c2 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:14.381929 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b94eb6-26c4-48d5-9112-54fcb9dbe9c2 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:14.382292 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b94eb6-26c4-48d5-9112-54fcb9dbe9c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:14.411666 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-d5b94eb6-26c4-48d5-9112-54fcb9dbe9c2 None None] MFA Rules not processed for user `a72376ecb8dc425e821b06b24bd6b9e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:40:14.412912 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-d5b94eb6-26c4-48d5-9112-54fcb9dbe9c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:14.429065 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f30fff7-c470-48da-b3f5-76397d021c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:14.429715 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f30fff7-c470-48da-b3f5-76397d021c71 admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:14.431052 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f30fff7-c470-48da-b3f5-76397d021c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:14.431052 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f30fff7-c470-48da-b3f5-76397d021c71 admin admin] PATH_INFO: `/v3/projects` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:14.431254 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f30fff7-c470-48da-b3f5-76397d021c71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:14.433228 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f30fff7-c470-48da-b3f5-76397d021c71 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:14.485135 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 956/1930] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:40:14 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:40:14.486543 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b94eb6-26c4-48d5-9112-54fcb9dbe9c2 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:14.493730 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-d5b94eb6-26c4-48d5-9112-54fcb9dbe9c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:14.543051 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 975/1931] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:40:14 2022] POST /identity/v3/projects => generated 387 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:40:14.543400 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f30fff7-c470-48da-b3f5-76397d021c71 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:14.560049 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b94eb6-26c4-48d5-9112-54fcb9dbe9c2 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:14.614182 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f30fff7-c470-48da-b3f5-76397d021c71 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:14.617367 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cee0a0-81a3-4dd8-9c35-81680a3a93ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d81f597cb44c56a966537089bcce4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:14.618086 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cee0a0-81a3-4dd8-9c35-81680a3a93ea service cinder] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:14.618372 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cee0a0-81a3-4dd8-9c35-81680a3a93ea service cinder] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:14.618653 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cee0a0-81a3-4dd8-9c35-81680a3a93ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:14.620474 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-a9cee0a0-81a3-4dd8-9c35-81680a3a93ea service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:14.621464 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a7cff6-ddc6-43c0-8f16-a1ac4cdff963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:14.622292 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a7cff6-ddc6-43c0-8f16-a1ac4cdff963 admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:14.622580 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a7cff6-ddc6-43c0-8f16-a1ac4cdff963 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:14.622863 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a7cff6-ddc6-43c0-8f16-a1ac4cdff963 admin admin] PATH_INFO: `/v3/projects` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:14.625190 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a7cff6-ddc6-43c0-8f16-a1ac4cdff963 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:14.627226 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a7cff6-ddc6-43c0-8f16-a1ac4cdff963 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:14.661042 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:40:14.711445 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cee0a0-81a3-4dd8-9c35-81680a3a93ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:14.713411 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cee0a0-81a3-4dd8-9c35-81680a3a93ea service cinder] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:14.723139 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 957/1932] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:40:14 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:40:14.729421 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cee0a0-81a3-4dd8-9c35-81680a3a93ea service cinder] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:14.742054 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cee0a0-81a3-4dd8-9c35-81680a3a93ea service cinder] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:14.768492 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 976/1933] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:40:14 2022] POST /identity/v3/projects => generated 401 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:40:14.770976 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a7cff6-ddc6-43c0-8f16-a1ac4cdff963 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:14.773431 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ee45ae-2f3e-47b6-83be-a6e542315a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:14.774221 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ee45ae-2f3e-47b6-83be-a6e542315a80 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:14.774569 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ee45ae-2f3e-47b6-83be-a6e542315a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:14.774910 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ee45ae-2f3e-47b6-83be-a6e542315a80 admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:14.775703 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ee45ae-2f3e-47b6-83be-a6e542315a80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:14.778303 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ee45ae-2f3e-47b6-83be-a6e542315a80 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:14.871486 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a7cff6-ddc6-43c0-8f16-a1ac4cdff963 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:14.879740 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbfa128-49b0-487c-bb06-359da0c86cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:14.880476 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbfa128-49b0-487c-bb06-359da0c86cab admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:14.881064 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbfa128-49b0-487c-bb06-359da0c86cab admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:14.881511 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbfa128-49b0-487c-bb06-359da0c86cab admin admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:14.882330 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbfa128-49b0-487c-bb06-359da0c86cab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:14.887668 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbfa128-49b0-487c-bb06-359da0c86cab admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:14.922387 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 958/1934] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:40:14 2022] POST /identity/v3/users => generated 294 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:40:14.927491 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ee45ae-2f3e-47b6-83be-a6e542315a80 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:14.975431 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ee45ae-2f3e-47b6-83be-a6e542315a80 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:14.987318 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 977/1935] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:40:14 2022] POST /identity/v3/users => generated 307 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:40:14.987447 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4599ead-b886-421e-ab5d-3d3d4ccf7da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:14.987447 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4599ead-b886-421e-ab5d-3d3d4ccf7da3 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:14.987696 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4599ead-b886-421e-ab5d-3d3d4ccf7da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:14.988129 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4599ead-b886-421e-ab5d-3d3d4ccf7da3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:14.989282 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbfa128-49b0-487c-bb06-359da0c86cab admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:14.989410 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4599ead-b886-421e-ab5d-3d3d4ccf7da3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:14.991703 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4599ead-b886-421e-ab5d-3d3d4ccf7da3 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:15.013807 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 959/1936] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:40:14 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:40:15.016892 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbfa128-49b0-487c-bb06-359da0c86cab admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:15.017011 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4599ead-b886-421e-ab5d-3d3d4ccf7da3 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:15.023628 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47061350-71f8-4d9a-ac6d-e66ddf7a18e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:15.024689 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47061350-71f8-4d9a-ac6d-e66ddf7a18e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:15.025128 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47061350-71f8-4d9a-ac6d-e66ddf7a18e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:15.025729 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47061350-71f8-4d9a-ac6d-e66ddf7a18e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:15.027169 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47061350-71f8-4d9a-ac6d-e66ddf7a18e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:15.027771 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4599ead-b886-421e-ab5d-3d3d4ccf7da3 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:15.030262 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47061350-71f8-4d9a-ac6d-e66ddf7a18e8 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:15.033808 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eee9a3e-4a80-4a45-9f07-06c86db0e315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:15.034601 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee9a3e-4a80-4a45-9f07-06c86db0e315 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:15.035793 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee9a3e-4a80-4a45-9f07-06c86db0e315 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:15.036180 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee9a3e-4a80-4a45-9f07-06c86db0e315 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:15.037144 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eee9a3e-4a80-4a45-9f07-06c86db0e315 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:15.038863 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eee9a3e-4a80-4a45-9f07-06c86db0e315 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:15.046776 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 978/1937] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:40:14 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:40:15.052156 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47061350-71f8-4d9a-ac6d-e66ddf7a18e8 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:15.057928 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 960/1938] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:40:15 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:40:15.064978 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47061350-71f8-4d9a-ac6d-e66ddf7a18e8 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:15.065563 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eee9a3e-4a80-4a45-9f07-06c86db0e315 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:15.072501 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b64184c-6575-4781-91ae-2c6e1c0463fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:15.074146 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b64184c-6575-4781-91ae-2c6e1c0463fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:15.074889 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b64184c-6575-4781-91ae-2c6e1c0463fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:15.075338 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b64184c-6575-4781-91ae-2c6e1c0463fd admin admin] PATH_INFO: `/v3/projects/0a052779b5ab475f958b7baf183d22f8/users/3e8b3032ec5f45c38ba6d7f246298ab2/roles/9baff2b18f2649f7b757ff2e1f63c853` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:15.078257 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eee9a3e-4a80-4a45-9f07-06c86db0e315 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:15.084055 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4e3a71-4fea-49c8-aa10-0720a62627b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:15.085008 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4e3a71-4fea-49c8-aa10-0720a62627b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:15.085294 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4e3a71-4fea-49c8-aa10-0720a62627b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:15.085682 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4e3a71-4fea-49c8-aa10-0720a62627b7 admin admin] PATH_INFO: `/v3/projects/c7e40df106114d96ba394711d42b234b/users/c6a440cf863549d29383aa9f5efcf045/roles/3771341b3be641139e752aab690397a4` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:15.108424 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b64184c-6575-4781-91ae-2c6e1c0463fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a052779b5ab475f958b7baf183d22f8, user_id=3e8b3032ec5f45c38ba6d7f246298ab2, role_id=9baff2b18f2649f7b757ff2e1f63c853)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:15.108645 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4e3a71-4fea-49c8-aa10-0720a62627b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7e40df106114d96ba394711d42b234b, user_id=c6a440cf863549d29383aa9f5efcf045, role_id=3771341b3be641139e752aab690397a4)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:15.108731 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b64184c-6575-4781-91ae-2c6e1c0463fd admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:15.108802 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4e3a71-4fea-49c8-aa10-0720a62627b7 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:15.195419 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 979/1939] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:40:15 2022] PUT /identity/v3/projects/0a052779b5ab475f958b7baf183d22f8/users/3e8b3032ec5f45c38ba6d7f246298ab2/roles/9baff2b18f2649f7b757ff2e1f63c853 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:40:15.195549 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 961/1940] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:40:15 2022] PUT /identity/v3/projects/c7e40df106114d96ba394711d42b234b/users/c6a440cf863549d29383aa9f5efcf045/roles/3771341b3be641139e752aab690397a4 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:40:15.199419 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b64184c-6575-4781-91ae-2c6e1c0463fd admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:15.209431 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d0b8f5-b677-4af4-8e8b-2c2d47b74d4e None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:15.209707 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d0b8f5-b677-4af4-8e8b-2c2d47b74d4e None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:15.209991 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d0b8f5-b677-4af4-8e8b-2c2d47b74d4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:15.237341 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-f8d0b8f5-b677-4af4-8e8b-2c2d47b74d4e None None] MFA Rules not processed for user `c6a440cf863549d29383aa9f5efcf045`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:40:15.238579 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-f8d0b8f5-b677-4af4-8e8b-2c2d47b74d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:15.278496 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b64184c-6575-4781-91ae-2c6e1c0463fd admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:15.320616 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 962/1941] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:40:15 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:40:15.321530 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d0b8f5-b677-4af4-8e8b-2c2d47b74d4e None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:15.349470 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244ec505-bac6-48d9-bc56-f406b14b63d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:15.352547 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244ec505-bac6-48d9-bc56-f406b14b63d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:15.352547 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244ec505-bac6-48d9-bc56-f406b14b63d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:15.352547 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244ec505-bac6-48d9-bc56-f406b14b63d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:15.352547 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244ec505-bac6-48d9-bc56-f406b14b63d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:15.353731 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244ec505-bac6-48d9-bc56-f406b14b63d5 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:15.368767 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 980/1942] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:40:15 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:40:15.374863 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:DELETE=3,keystone:SELECT=92,keystone:INSERT=5 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:40:15.376557 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244ec505-bac6-48d9-bc56-f406b14b63d5 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:15.398499 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244ec505-bac6-48d9-bc56-f406b14b63d5 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:15.403400 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d0b8f5-b677-4af4-8e8b-2c2d47b74d4e None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:15.403673 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b390eab-8c0a-4474-80e5-c032e409ff62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:15.405114 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b390eab-8c0a-4474-80e5-c032e409ff62 admin admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:15.405495 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b390eab-8c0a-4474-80e5-c032e409ff62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:15.405782 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b390eab-8c0a-4474-80e5-c032e409ff62 admin admin] PATH_INFO: `/v3/projects/0a052779b5ab475f958b7baf183d22f8/users/3e8b3032ec5f45c38ba6d7f246298ab2/roles/9baff2b18f2649f7b757ff2e1f63c853` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:15.407819 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b390eab-8c0a-4474-80e5-c032e409ff62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a052779b5ab475f958b7baf183d22f8, user_id=3e8b3032ec5f45c38ba6d7f246298ab2, role_id=9baff2b18f2649f7b757ff2e1f63c853)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:15.410792 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b390eab-8c0a-4474-80e5-c032e409ff62 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:15.410969 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261dd264-bafc-4327-b6bf-a8c102e63ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:15.411708 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261dd264-bafc-4327-b6bf-a8c102e63ccd admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:15.412263 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261dd264-bafc-4327-b6bf-a8c102e63ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:15.412768 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261dd264-bafc-4327-b6bf-a8c102e63ccd admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:15.413598 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261dd264-bafc-4327-b6bf-a8c102e63ccd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:15.415818 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261dd264-bafc-4327-b6bf-a8c102e63ccd admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:15.424018 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 981/1943] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:40:15 2022] PUT /identity/v3/projects/0a052779b5ab475f958b7baf183d22f8/users/3e8b3032ec5f45c38ba6d7f246298ab2/roles/9baff2b18f2649f7b757ff2e1f63c853 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:40:15.447545 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806bdbd9-2bbc-401d-b859-b140af53eb58 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:15.448244 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806bdbd9-2bbc-401d-b859-b140af53eb58 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:15.448773 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806bdbd9-2bbc-401d-b859-b140af53eb58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:15.488512 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-806bdbd9-2bbc-401d-b859-b140af53eb58 None None] MFA Rules not processed for user `3e8b3032ec5f45c38ba6d7f246298ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:40:15.489726 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-806bdbd9-2bbc-401d-b859-b140af53eb58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:15.501684 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 963/1944] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:40:15 2022] POST /identity/v3/projects => generated 387 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:40:15.506160 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261dd264-bafc-4327-b6bf-a8c102e63ccd admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:15.648049 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 982/1945] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:40:15 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:40:15.651723 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261dd264-bafc-4327-b6bf-a8c102e63ccd admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:15.661670 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b931094-591f-4902-91be-c907c2ff59fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:15.662643 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b931094-591f-4902-91be-c907c2ff59fc admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:15.663187 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b931094-591f-4902-91be-c907c2ff59fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:15.663631 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b931094-591f-4902-91be-c907c2ff59fc admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:15.664619 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b931094-591f-4902-91be-c907c2ff59fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:15.667339 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b931094-591f-4902-91be-c907c2ff59fc admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:15.673642 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:40:15.774675 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 964/1946] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:40:15 2022] POST /identity/v3/users => generated 300 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:40:15.779464 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806bdbd9-2bbc-401d-b859-b140af53eb58 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:15.962533 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b931094-591f-4902-91be-c907c2ff59fc admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:15.984311 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806bdbd9-2bbc-401d-b859-b140af53eb58 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:15.993878 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6cd50b-a62c-40f2-8a1f-356e229d42cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:15.995264 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6cd50b-a62c-40f2-8a1f-356e229d42cb admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:15.995581 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6cd50b-a62c-40f2-8a1f-356e229d42cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:15.996111 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6cd50b-a62c-40f2-8a1f-356e229d42cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:15.997278 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6cd50b-a62c-40f2-8a1f-356e229d42cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:16.002551 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6cd50b-a62c-40f2-8a1f-356e229d42cb admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:16.023743 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 983/1947] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:40:15 2022] GET /identity/v3/roles => generated 1420 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:40:16.027777 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6cd50b-a62c-40f2-8a1f-356e229d42cb admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:16.079266 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6cd50b-a62c-40f2-8a1f-356e229d42cb admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:16.096020 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed11e8a-64ba-44ff-8bca-d4294fb54127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:16.097806 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed11e8a-64ba-44ff-8bca-d4294fb54127 admin admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:16.098094 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed11e8a-64ba-44ff-8bca-d4294fb54127 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:16.101027 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed11e8a-64ba-44ff-8bca-d4294fb54127 admin admin] PATH_INFO: `/v3/projects/ee657a66767a428b90063c505e7010b1/users/2487d65f203842b1bd4742dbb844e122/roles/9baff2b18f2649f7b757ff2e1f63c853` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:16.123958 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:40:16.127698 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b931094-591f-4902-91be-c907c2ff59fc admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:16.133410 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed11e8a-64ba-44ff-8bca-d4294fb54127 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee657a66767a428b90063c505e7010b1, user_id=2487d65f203842b1bd4742dbb844e122, role_id=9baff2b18f2649f7b757ff2e1f63c853)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:16.134323 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bbfc4f-ff36-4dc4-bff4-9ebcdb8d220c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d81f597cb44c56a966537089bcce4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:16.135188 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bbfc4f-ff36-4dc4-bff4-9ebcdb8d220c service cinder] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:16.135484 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bbfc4f-ff36-4dc4-bff4-9ebcdb8d220c service cinder] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:16.135842 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bbfc4f-ff36-4dc4-bff4-9ebcdb8d220c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:16.140182 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed11e8a-64ba-44ff-8bca-d4294fb54127 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:16.215761 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bbfc4f-ff36-4dc4-bff4-9ebcdb8d220c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:16.218310 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bbfc4f-ff36-4dc4-bff4-9ebcdb8d220c service cinder] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:16.234356 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 965/1948] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:40:15 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 14:40:16.245936 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 984/1949] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:40:16 2022] PUT /identity/v3/projects/ee657a66767a428b90063c505e7010b1/users/2487d65f203842b1bd4742dbb844e122/roles/9baff2b18f2649f7b757ff2e1f63c853 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:40:16.248629 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed11e8a-64ba-44ff-8bca-d4294fb54127 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:16.335814 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:40:16.381911 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed11e8a-64ba-44ff-8bca-d4294fb54127 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:16.390185 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a01d639-88d0-413f-9345-b98545105102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:16.391011 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a01d639-88d0-413f-9345-b98545105102 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:16.391356 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a01d639-88d0-413f-9345-b98545105102 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:16.391731 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a01d639-88d0-413f-9345-b98545105102 admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:16.393409 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a01d639-88d0-413f-9345-b98545105102 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:16.396407 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a01d639-88d0-413f-9345-b98545105102 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:16.420118 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 985/1950] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:40:16 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:40:16.424654 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bbfc4f-ff36-4dc4-bff4-9ebcdb8d220c service cinder] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:16.438521 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bbfc4f-ff36-4dc4-bff4-9ebcdb8d220c service cinder] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:16.445729 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2cbf48-c3bb-4e69-a0b3-80226549cc7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:16.446680 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2cbf48-c3bb-4e69-a0b3-80226549cc7f admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:16.446976 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2cbf48-c3bb-4e69-a0b3-80226549cc7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:16.447246 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2cbf48-c3bb-4e69-a0b3-80226549cc7f admin admin] PATH_INFO: `/v3/projects/ee657a66767a428b90063c505e7010b1/users/2487d65f203842b1bd4742dbb844e122/roles/9baff2b18f2649f7b757ff2e1f63c853` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:16.450339 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2cbf48-c3bb-4e69-a0b3-80226549cc7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee657a66767a428b90063c505e7010b1, user_id=2487d65f203842b1bd4742dbb844e122, role_id=9baff2b18f2649f7b757ff2e1f63c853)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:16.457216 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2cbf48-c3bb-4e69-a0b3-80226549cc7f admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:16.475895 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 966/1951] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:40:16 2022] PUT /identity/v3/projects/ee657a66767a428b90063c505e7010b1/users/2487d65f203842b1bd4742dbb844e122/roles/9baff2b18f2649f7b757ff2e1f63c853 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:40:16.478102 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:40:16.486060 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c0327d-7b13-479b-b064-74844aea3375 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:16.490679 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c0327d-7b13-479b-b064-74844aea3375 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:16.490679 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c0327d-7b13-479b-b064-74844aea3375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:16.507128 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2cbf48-c3bb-4e69-a0b3-80226549cc7f admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:16.519334 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-24c0327d-7b13-479b-b064-74844aea3375 None None] MFA Rules not processed for user `2487d65f203842b1bd4742dbb844e122`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:40:16.520708 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-24c0327d-7b13-479b-b064-74844aea3375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:16.615800 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2cbf48-c3bb-4e69-a0b3-80226549cc7f admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:16.620273 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea89bad4-37e5-45f3-9ebf-23ab9da0ad54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d81f597cb44c56a966537089bcce4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:16.620954 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea89bad4-37e5-45f3-9ebf-23ab9da0ad54 service cinder] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:16.621260 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea89bad4-37e5-45f3-9ebf-23ab9da0ad54 service cinder] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:16.621594 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea89bad4-37e5-45f3-9ebf-23ab9da0ad54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:16.630174 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 986/1952] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:40:16 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:40:16.645945 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c0327d-7b13-479b-b064-74844aea3375 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:16.657384 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c0327d-7b13-479b-b064-74844aea3375 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:16.662212 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f10f15-10b8-492a-97c9-9f050092d86d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d81f597cb44c56a966537089bcce4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:16.662965 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f10f15-10b8-492a-97c9-9f050092d86d service cinder] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:16.663179 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f10f15-10b8-492a-97c9-9f050092d86d service cinder] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:16.663443 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f10f15-10b8-492a-97c9-9f050092d86d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:16.676861 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f10f15-10b8-492a-97c9-9f050092d86d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:16.678865 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f10f15-10b8-492a-97c9-9f050092d86d service cinder] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:16.697909 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea89bad4-37e5-45f3-9ebf-23ab9da0ad54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:16.699625 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea89bad4-37e5-45f3-9ebf-23ab9da0ad54 service cinder] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:16.700210 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 987/1953] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:40:16 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 14:40:16.712344 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 967/1954] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:40:16 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 14:40:16.805964 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:40:16.815036 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f10f15-10b8-492a-97c9-9f050092d86d service cinder] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:17.108709 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f10f15-10b8-492a-97c9-9f050092d86d service cinder] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:17.108709 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37938452-35a6-4424-b2af-8f9eeb2c5c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:17.108709 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37938452-35a6-4424-b2af-8f9eeb2c5c83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:17.108709 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37938452-35a6-4424-b2af-8f9eeb2c5c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:17.111822 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37938452-35a6-4424-b2af-8f9eeb2c5c83 admin admin] PATH_INFO: `/v3/users/c6a440cf863549d29383aa9f5efcf045` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:17.111822 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37938452-35a6-4424-b2af-8f9eeb2c5c83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6a440cf863549d29383aa9f5efcf045)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:17.111822 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37938452-35a6-4424-b2af-8f9eeb2c5c83 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:17.394372 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-37938452-35a6-4424-b2af-8f9eeb2c5c83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6a440cf863549d29383aa9f5efcf045'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:17.395695 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-37938452-35a6-4424-b2af-8f9eeb2c5c83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6a440cf863549d29383aa9f5efcf045'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:17.396525 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-37938452-35a6-4424-b2af-8f9eeb2c5c83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6a440cf863549d29383aa9f5efcf045'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:17.410217 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-37938452-35a6-4424-b2af-8f9eeb2c5c83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6a440cf863549d29383aa9f5efcf045'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:17.447485 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-37938452-35a6-4424-b2af-8f9eeb2c5c83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6a440cf863549d29383aa9f5efcf045'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:17.537279 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 988/1955] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:40:16 2022] DELETE /identity/v3/users/c6a440cf863549d29383aa9f5efcf045 => generated 0 bytes in 724 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:40:17.546946 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deb517f-0bd0-4117-bea9-1de572dd302e None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:17.547513 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deb517f-0bd0-4117-bea9-1de572dd302e None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:17.547513 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deb517f-0bd0-4117-bea9-1de572dd302e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:17.580671 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-1deb517f-0bd0-4117-bea9-1de572dd302e None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:40:17.582537 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-1deb517f-0bd0-4117-bea9-1de572dd302e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:17.660295 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 968/1956] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:40:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:40:17.673670 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37938452-35a6-4424-b2af-8f9eeb2c5c83 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:17.673670 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-37938452-35a6-4424-b2af-8f9eeb2c5c83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:17.744924 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37938452-35a6-4424-b2af-8f9eeb2c5c83 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:17.749955 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c729263-6d0c-45ed-ac9a-fe54999648ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:17.750744 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c729263-6d0c-45ed-ac9a-fe54999648ce service neutron] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:17.751043 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c729263-6d0c-45ed-ac9a-fe54999648ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:17.751387 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c729263-6d0c-45ed-ac9a-fe54999648ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:17.762039 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c729263-6d0c-45ed-ac9a-fe54999648ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:17.763901 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c729263-6d0c-45ed-ac9a-fe54999648ce service neutron] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:17.775267 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:40:17.797737 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 989/1957] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:40:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:40:17.991543 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:40:18.675659 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deb517f-0bd0-4117-bea9-1de572dd302e None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:18.677245 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-1deb517f-0bd0-4117-bea9-1de572dd302e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:18.700724 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:40:18.706482 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deb517f-0bd0-4117-bea9-1de572dd302e None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:18.722029 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc8749c-ed96-4418-9131-7806b2226d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:18.726095 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc8749c-ed96-4418-9131-7806b2226d25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:18.728835 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc8749c-ed96-4418-9131-7806b2226d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:18.731654 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc8749c-ed96-4418-9131-7806b2226d25 admin admin] PATH_INFO: `/v3/projects/c7e40df106114d96ba394711d42b234b` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:18.743802 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc8749c-ed96-4418-9131-7806b2226d25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7e40df106114d96ba394711d42b234b)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:18.750858 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc8749c-ed96-4418-9131-7806b2226d25 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:18.988908 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-6cc8749c-ed96-4418-9131-7806b2226d25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7e40df106114d96ba394711d42b234b', 'request_id': 'req-6cc8749c-ed96-4418-9131-7806b2226d25'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:19.043682 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:40:19.107915 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-6cc8749c-ed96-4418-9131-7806b2226d25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7e40df106114d96ba394711d42b234b', 'request_id': 'req-6cc8749c-ed96-4418-9131-7806b2226d25'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:19.119236 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-6cc8749c-ed96-4418-9131-7806b2226d25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7e40df106114d96ba394711d42b234b', 'request_id': 'req-6cc8749c-ed96-4418-9131-7806b2226d25'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:19.135157 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-6cc8749c-ed96-4418-9131-7806b2226d25 admin admin] The token cache is being invalidate because project c7e40df106114d96ba394711d42b234b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:40:19.135869 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-6cc8749c-ed96-4418-9131-7806b2226d25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:19.137205 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-6cc8749c-ed96-4418-9131-7806b2226d25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:19.143750 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 969/1958] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:40:18 2022] DELETE /identity/v3/projects/c7e40df106114d96ba394711d42b234b => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:40:19.144917 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c729263-6d0c-45ed-ac9a-fe54999648ce service neutron] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:19.155877 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c729263-6d0c-45ed-ac9a-fe54999648ce service neutron] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:19.226202 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6788f6a5-c460-4e78-8700-8760666a5135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:19.229789 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788f6a5-c460-4e78-8700-8760666a5135 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:19.229789 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788f6a5-c460-4e78-8700-8760666a5135 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:19.229789 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788f6a5-c460-4e78-8700-8760666a5135 admin admin] PATH_INFO: `/v3/users/2487d65f203842b1bd4742dbb844e122` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:19.229789 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6788f6a5-c460-4e78-8700-8760666a5135 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2487d65f203842b1bd4742dbb844e122)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:19.232159 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6788f6a5-c460-4e78-8700-8760666a5135 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:19.455959 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-6788f6a5-c460-4e78-8700-8760666a5135 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2487d65f203842b1bd4742dbb844e122'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:19.456684 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-6788f6a5-c460-4e78-8700-8760666a5135 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2487d65f203842b1bd4742dbb844e122'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:19.457386 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-6788f6a5-c460-4e78-8700-8760666a5135 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2487d65f203842b1bd4742dbb844e122'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:19.488381 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-6788f6a5-c460-4e78-8700-8760666a5135 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2487d65f203842b1bd4742dbb844e122'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:19.532740 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-6788f6a5-c460-4e78-8700-8760666a5135 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2487d65f203842b1bd4742dbb844e122'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:19.627017 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 990/1959] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:40:19 2022] DELETE /identity/v3/users/2487d65f203842b1bd4742dbb844e122 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:40:20.048451 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:40:20.311118 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc8749c-ed96-4418-9131-7806b2226d25 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:20.313870 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-6cc8749c-ed96-4418-9131-7806b2226d25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:20.353593 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc8749c-ed96-4418-9131-7806b2226d25 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:20.419096 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:40:20.439143 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845ff7e5-442f-4be6-89cd-144e7bf0e325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:20.440350 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845ff7e5-442f-4be6-89cd-144e7bf0e325 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:20.440735 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845ff7e5-442f-4be6-89cd-144e7bf0e325 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:20.441072 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845ff7e5-442f-4be6-89cd-144e7bf0e325 admin admin] PATH_INFO: `/v3/projects/ee657a66767a428b90063c505e7010b1` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:20.442813 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845ff7e5-442f-4be6-89cd-144e7bf0e325 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee657a66767a428b90063c505e7010b1)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:20.445995 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845ff7e5-442f-4be6-89cd-144e7bf0e325 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:20.565580 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-845ff7e5-442f-4be6-89cd-144e7bf0e325 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee657a66767a428b90063c505e7010b1', 'request_id': 'req-845ff7e5-442f-4be6-89cd-144e7bf0e325'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:20.615376 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-845ff7e5-442f-4be6-89cd-144e7bf0e325 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee657a66767a428b90063c505e7010b1', 'request_id': 'req-845ff7e5-442f-4be6-89cd-144e7bf0e325'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:20.624481 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-845ff7e5-442f-4be6-89cd-144e7bf0e325 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee657a66767a428b90063c505e7010b1', 'request_id': 'req-845ff7e5-442f-4be6-89cd-144e7bf0e325'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:20.644235 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-845ff7e5-442f-4be6-89cd-144e7bf0e325 admin admin] The token cache is being invalidate because project ee657a66767a428b90063c505e7010b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:40:20.644537 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-845ff7e5-442f-4be6-89cd-144e7bf0e325 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:20.648754 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-845ff7e5-442f-4be6-89cd-144e7bf0e325 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:20.648950 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 970/1960] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:40:20 2022] DELETE /identity/v3/projects/ee657a66767a428b90063c505e7010b1 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:40:20.654374 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed7340a-e4fe-4ac1-8263-d19b5b3f7e21 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:20.654696 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed7340a-e4fe-4ac1-8263-d19b5b3f7e21 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:20.654979 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed7340a-e4fe-4ac1-8263-d19b5b3f7e21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:20.681208 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-fed7340a-e4fe-4ac1-8263-d19b5b3f7e21 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:40:20.681208 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-fed7340a-e4fe-4ac1-8263-d19b5b3f7e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:20.771995 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 991/1961] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:40:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:40:20.780242 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b5921c-0c27-464a-9272-47d90d1da321 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:20.780599 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b5921c-0c27-464a-9272-47d90d1da321 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:20.780982 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b5921c-0c27-464a-9272-47d90d1da321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:20.802949 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-58b5921c-0c27-464a-9272-47d90d1da321 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:40:20.804185 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-58b5921c-0c27-464a-9272-47d90d1da321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:20.814070 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 971/1962] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:40:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:40:20.814261 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed7340a-e4fe-4ac1-8263-d19b5b3f7e21 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:20.815731 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-fed7340a-e4fe-4ac1-8263-d19b5b3f7e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:20.818039 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b5921c-0c27-464a-9272-47d90d1da321 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:20.829347 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed7340a-e4fe-4ac1-8263-d19b5b3f7e21 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:20.831779 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b5921c-0c27-464a-9272-47d90d1da321 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:20.835053 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56448d1-6d03-42fb-8217-d1cef2f9eb54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:20.835992 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56448d1-6d03-42fb-8217-d1cef2f9eb54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:20.836302 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56448d1-6d03-42fb-8217-d1cef2f9eb54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:20.836623 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56448d1-6d03-42fb-8217-d1cef2f9eb54 admin admin] PATH_INFO: `/v3/users/1eac7fccb72b4328bd339929472abe0e` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:20.837458 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246ea57d-449e-4e66-b526-4528610a42a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:20.837984 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56448d1-6d03-42fb-8217-d1cef2f9eb54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1eac7fccb72b4328bd339929472abe0e)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:20.838168 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246ea57d-449e-4e66-b526-4528610a42a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:20.838455 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246ea57d-449e-4e66-b526-4528610a42a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:20.838733 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246ea57d-449e-4e66-b526-4528610a42a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:20.839907 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246ea57d-449e-4e66-b526-4528610a42a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:40:20.839907 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246ea57d-449e-4e66-b526-4528610a42a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:20.840134 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56448d1-6d03-42fb-8217-d1cef2f9eb54 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:20.841628 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246ea57d-449e-4e66-b526-4528610a42a4 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:20.853479 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 972/1963] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jun 7 14:40:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:40:20.860089 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246ea57d-449e-4e66-b526-4528610a42a4 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:20.870975 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246ea57d-449e-4e66-b526-4528610a42a4 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:20.876710 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a3e2fd-39c4-4f3c-9904-4c4b104983e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:20.877379 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a3e2fd-39c4-4f3c-9904-4c4b104983e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:20.877668 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a3e2fd-39c4-4f3c-9904-4c4b104983e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:20.877950 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a3e2fd-39c4-4f3c-9904-4c4b104983e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:20.878682 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a3e2fd-39c4-4f3c-9904-4c4b104983e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:20.880659 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a3e2fd-39c4-4f3c-9904-4c4b104983e8 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:21.050358 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 973/1964] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:40:20 2022] POST /identity/v3/projects => generated 409 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:40:21.054716 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a3e2fd-39c4-4f3c-9904-4c4b104983e8 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:21.080125 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:40:21.095228 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-c56448d1-6d03-42fb-8217-d1cef2f9eb54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1eac7fccb72b4328bd339929472abe0e'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:21.096032 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-c56448d1-6d03-42fb-8217-d1cef2f9eb54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1eac7fccb72b4328bd339929472abe0e'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:21.096681 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-c56448d1-6d03-42fb-8217-d1cef2f9eb54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1eac7fccb72b4328bd339929472abe0e'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:21.103918 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-c56448d1-6d03-42fb-8217-d1cef2f9eb54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1eac7fccb72b4328bd339929472abe0e'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:21.116860 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-c56448d1-6d03-42fb-8217-d1cef2f9eb54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1eac7fccb72b4328bd339929472abe0e'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:21.129386 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a3e2fd-39c4-4f3c-9904-4c4b104983e8 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:21.134291 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e93cfa-a5b7-4261-af11-93b99dabc348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:21.134967 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e93cfa-a5b7-4261-af11-93b99dabc348 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:21.135257 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e93cfa-a5b7-4261-af11-93b99dabc348 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:21.135538 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e93cfa-a5b7-4261-af11-93b99dabc348 admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:21.136267 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e93cfa-a5b7-4261-af11-93b99dabc348 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:21.138514 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e93cfa-a5b7-4261-af11-93b99dabc348 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:21.186713 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 992/1965] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:40:20 2022] DELETE /identity/v3/users/1eac7fccb72b4328bd339929472abe0e => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:40:21.191416 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ec2cbc-270f-4028-a101-af495c243606 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:21.191717 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ec2cbc-270f-4028-a101-af495c243606 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:21.191997 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ec2cbc-270f-4028-a101-af495c243606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:21.210311 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-48ec2cbc-270f-4028-a101-af495c243606 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:40:21.211529 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-48ec2cbc-270f-4028-a101-af495c243606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:21.238565 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 974/1966] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:40:21 2022] POST /identity/v3/users => generated 305 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:40:21.243165 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e93cfa-a5b7-4261-af11-93b99dabc348 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:21.246174 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-b7e93cfa-a5b7-4261-af11-93b99dabc348 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:21.261611 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e93cfa-a5b7-4261-af11-93b99dabc348 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:21.280579 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 993/1967] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:40:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:40:21.290713 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ec2cbc-270f-4028-a101-af495c243606 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:21.292235 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-48ec2cbc-270f-4028-a101-af495c243606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:21.337771 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6543354-94d2-4602-ab30-04d99d91c2e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:21.338811 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6543354-94d2-4602-ab30-04d99d91c2e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:21.339261 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6543354-94d2-4602-ab30-04d99d91c2e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:21.339683 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6543354-94d2-4602-ab30-04d99d91c2e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:21.340657 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6543354-94d2-4602-ab30-04d99d91c2e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:21.344279 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6543354-94d2-4602-ab30-04d99d91c2e7 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:21.361738 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 975/1968] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:40:21 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 14:40:21.365996 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6543354-94d2-4602-ab30-04d99d91c2e7 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:21.367523 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ec2cbc-270f-4028-a101-af495c243606 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:21.372490 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4024bc-fd52-493c-b45f-c3f82eb2d1a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:21.373193 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4024bc-fd52-493c-b45f-c3f82eb2d1a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:21.377642 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4024bc-fd52-493c-b45f-c3f82eb2d1a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:21.377642 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4024bc-fd52-493c-b45f-c3f82eb2d1a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:21.380607 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6543354-94d2-4602-ab30-04d99d91c2e7 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:21.385896 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4024bc-fd52-493c-b45f-c3f82eb2d1a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:21.387354 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4024bc-fd52-493c-b45f-c3f82eb2d1a0 service neutron] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:21.400870 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8060e72-170e-42af-b76f-8334f89231e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:21.412087 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8060e72-170e-42af-b76f-8334f89231e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:21.412087 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8060e72-170e-42af-b76f-8334f89231e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:21.412087 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8060e72-170e-42af-b76f-8334f89231e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:21.412087 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8060e72-170e-42af-b76f-8334f89231e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:21.412087 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8060e72-170e-42af-b76f-8334f89231e0 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:21.413024 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 994/1969] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:40:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 14:40:21.418991 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 976/1970] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:40:21 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 14:40:21.425474 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4024bc-fd52-493c-b45f-c3f82eb2d1a0 service neutron] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:21.436865 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4024bc-fd52-493c-b45f-c3f82eb2d1a0 service neutron] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:21.437369 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:40:21.445568 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93743b09-403f-4b57-b52e-458f0a6e59b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:21.446518 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93743b09-403f-4b57-b52e-458f0a6e59b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:21.452350 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93743b09-403f-4b57-b52e-458f0a6e59b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:21.452350 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93743b09-403f-4b57-b52e-458f0a6e59b7 admin admin] PATH_INFO: `/v3/projects/7123e05a6a4143e6b4a24f2d9322508c/users/ca63a39dcc00445b984d9455294f8ff1/roles/3771341b3be641139e752aab690397a4` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:21.475729 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:40:21.476929 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93743b09-403f-4b57-b52e-458f0a6e59b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7123e05a6a4143e6b4a24f2d9322508c, user_id=ca63a39dcc00445b984d9455294f8ff1, role_id=3771341b3be641139e752aab690397a4)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:21.480581 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93743b09-403f-4b57-b52e-458f0a6e59b7 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:21.552979 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 995/1971] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:40:21 2022] PUT /identity/v3/projects/7123e05a6a4143e6b4a24f2d9322508c/users/ca63a39dcc00445b984d9455294f8ff1/roles/3771341b3be641139e752aab690397a4 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:40:21.564442 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c800be82-8c98-497e-9dd0-dd4d68241977 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:21.564734 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c800be82-8c98-497e-9dd0-dd4d68241977 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:21.565013 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c800be82-8c98-497e-9dd0-dd4d68241977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:21.598867 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-c800be82-8c98-497e-9dd0-dd4d68241977 None None] MFA Rules not processed for user `ca63a39dcc00445b984d9455294f8ff1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:40:21.599988 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-c800be82-8c98-497e-9dd0-dd4d68241977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:21.707115 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 977/1972] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:40:21 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:40:21.711426 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93743b09-403f-4b57-b52e-458f0a6e59b7 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:21.804959 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93743b09-403f-4b57-b52e-458f0a6e59b7 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:21.812680 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5197ee29-bafb-4c6a-a6af-fc0cf2adc5e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:21.813659 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5197ee29-bafb-4c6a-a6af-fc0cf2adc5e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:21.815373 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5197ee29-bafb-4c6a-a6af-fc0cf2adc5e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:21.815373 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5197ee29-bafb-4c6a-a6af-fc0cf2adc5e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:21.815373 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5197ee29-bafb-4c6a-a6af-fc0cf2adc5e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:21.817617 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5197ee29-bafb-4c6a-a6af-fc0cf2adc5e0 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:21.900121 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:40:21.952332 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c800be82-8c98-497e-9dd0-dd4d68241977 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:21.959989 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 996/1973] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:40:21 2022] POST /identity/v3/projects => generated 409 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:40:21.960199 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-c800be82-8c98-497e-9dd0-dd4d68241977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:21.960296 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5197ee29-bafb-4c6a-a6af-fc0cf2adc5e0 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:21.981025 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c800be82-8c98-497e-9dd0-dd4d68241977 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:22.044290 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5197ee29-bafb-4c6a-a6af-fc0cf2adc5e0 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:22.049202 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8486b985-34d5-4842-bc87-5e1a9fae2359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:22.049860 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8486b985-34d5-4842-bc87-5e1a9fae2359 admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:22.050153 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8486b985-34d5-4842-bc87-5e1a9fae2359 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:22.050432 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8486b985-34d5-4842-bc87-5e1a9fae2359 admin admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:22.051122 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8486b985-34d5-4842-bc87-5e1a9fae2359 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:22.051677 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ef33d0-c526-4c4d-9f93-8e7130142d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:22.053117 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ef33d0-c526-4c4d-9f93-8e7130142d7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:22.053262 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8486b985-34d5-4842-bc87-5e1a9fae2359 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:22.053485 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ef33d0-c526-4c4d-9f93-8e7130142d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:22.053906 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ef33d0-c526-4c4d-9f93-8e7130142d7b admin admin] PATH_INFO: `/v3/projects/735825f6725d409e9aa7e442497a19bb` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:22.055224 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ef33d0-c526-4c4d-9f93-8e7130142d7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=735825f6725d409e9aa7e442497a19bb)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:22.057348 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ef33d0-c526-4c4d-9f93-8e7130142d7b admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:22.157214 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 997/1974] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:40:21 2022] POST /identity/v3/users => generated 311 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:40:22.161295 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8486b985-34d5-4842-bc87-5e1a9fae2359 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:22.164857 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:40:22.171202 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-25ef33d0-c526-4c4d-9f93-8e7130142d7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '735825f6725d409e9aa7e442497a19bb', 'request_id': 'req-25ef33d0-c526-4c4d-9f93-8e7130142d7b'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:22.171563 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8486b985-34d5-4842-bc87-5e1a9fae2359 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:22.183608 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:40:22.229779 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af67a98-2dcc-42dc-833f-088f98519343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:22.230463 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af67a98-2dcc-42dc-833f-088f98519343 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:22.230833 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af67a98-2dcc-42dc-833f-088f98519343 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:22.231187 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af67a98-2dcc-42dc-833f-088f98519343 admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:22.231950 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af67a98-2dcc-42dc-833f-088f98519343 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:22.233869 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af67a98-2dcc-42dc-833f-088f98519343 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:22.244688 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 998/1975] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:40:22 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:40:22.249019 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af67a98-2dcc-42dc-833f-088f98519343 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:22.254322 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-25ef33d0-c526-4c4d-9f93-8e7130142d7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '735825f6725d409e9aa7e442497a19bb', 'request_id': 'req-25ef33d0-c526-4c4d-9f93-8e7130142d7b'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:22.260536 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af67a98-2dcc-42dc-833f-088f98519343 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:22.266370 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-25ef33d0-c526-4c4d-9f93-8e7130142d7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '735825f6725d409e9aa7e442497a19bb', 'request_id': 'req-25ef33d0-c526-4c4d-9f93-8e7130142d7b'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:22.267245 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7038f082-fbe9-4d13-a36c-f3fe65fb03a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:22.268795 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7038f082-fbe9-4d13-a36c-f3fe65fb03a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:22.269871 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7038f082-fbe9-4d13-a36c-f3fe65fb03a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:22.269871 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7038f082-fbe9-4d13-a36c-f3fe65fb03a3 admin admin] PATH_INFO: `/v3/projects/4103c823578e4fc0a5a96b7a4e631855/users/a04507052803403ba5a5f29b416220fa/roles/9baff2b18f2649f7b757ff2e1f63c853` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:22.286268 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-25ef33d0-c526-4c4d-9f93-8e7130142d7b admin admin] The token cache is being invalidate because project 735825f6725d409e9aa7e442497a19bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:40:22.286844 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-25ef33d0-c526-4c4d-9f93-8e7130142d7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:22.287780 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-25ef33d0-c526-4c4d-9f93-8e7130142d7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:22.294745 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 978/1976] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:40:21 2022] DELETE /identity/v3/projects/735825f6725d409e9aa7e442497a19bb => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:40:22.296148 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ef33d0-c526-4c4d-9f93-8e7130142d7b admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:22.297942 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7038f082-fbe9-4d13-a36c-f3fe65fb03a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4103c823578e4fc0a5a96b7a4e631855, user_id=a04507052803403ba5a5f29b416220fa, role_id=9baff2b18f2649f7b757ff2e1f63c853)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:22.298364 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-25ef33d0-c526-4c4d-9f93-8e7130142d7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:22.300738 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7038f082-fbe9-4d13-a36c-f3fe65fb03a3 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:22.315596 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ef33d0-c526-4c4d-9f93-8e7130142d7b admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:22.321028 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b263f5a-68b8-4728-baa9-747785b8ceb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:22.321912 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b263f5a-68b8-4728-baa9-747785b8ceb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:22.322200 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b263f5a-68b8-4728-baa9-747785b8ceb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:22.322642 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b263f5a-68b8-4728-baa9-747785b8ceb7 admin admin] PATH_INFO: `/v3/users/d95a768ee823408dad5d9c831201a5fc` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:22.323877 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b263f5a-68b8-4728-baa9-747785b8ceb7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d95a768ee823408dad5d9c831201a5fc)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:22.325834 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b263f5a-68b8-4728-baa9-747785b8ceb7 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:22.385898 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 999/1977] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:40:22 2022] PUT /identity/v3/projects/4103c823578e4fc0a5a96b7a4e631855/users/a04507052803403ba5a5f29b416220fa/roles/9baff2b18f2649f7b757ff2e1f63c853 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:40:22.391032 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7038f082-fbe9-4d13-a36c-f3fe65fb03a3 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:22.392775 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-7038f082-fbe9-4d13-a36c-f3fe65fb03a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:22.460119 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7038f082-fbe9-4d13-a36c-f3fe65fb03a3 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:22.465603 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e175eae0-89f7-4bab-9561-8915e88a526a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:22.466288 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e175eae0-89f7-4bab-9561-8915e88a526a admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:22.466592 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e175eae0-89f7-4bab-9561-8915e88a526a admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:22.466878 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e175eae0-89f7-4bab-9561-8915e88a526a admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:22.468619 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e175eae0-89f7-4bab-9561-8915e88a526a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:22.469032 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-6b263f5a-68b8-4728-baa9-747785b8ceb7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd95a768ee823408dad5d9c831201a5fc'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:22.469803 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-6b263f5a-68b8-4728-baa9-747785b8ceb7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd95a768ee823408dad5d9c831201a5fc'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:22.470411 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e175eae0-89f7-4bab-9561-8915e88a526a admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:22.470533 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-6b263f5a-68b8-4728-baa9-747785b8ceb7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd95a768ee823408dad5d9c831201a5fc'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:22.486602 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-6b263f5a-68b8-4728-baa9-747785b8ceb7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd95a768ee823408dad5d9c831201a5fc'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:22.487211 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1000/1978] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:40:22 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 14:40:22.491147 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e175eae0-89f7-4bab-9561-8915e88a526a admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:22.493041 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-e175eae0-89f7-4bab-9561-8915e88a526a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:22.498172 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-6b263f5a-68b8-4728-baa9-747785b8ceb7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd95a768ee823408dad5d9c831201a5fc'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:22.511488 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e175eae0-89f7-4bab-9561-8915e88a526a admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:22.513469 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfd7ce1-daf7-4f69-bc3f-87f0e097c87e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:22.514924 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfd7ce1-daf7-4f69-bc3f-87f0e097c87e admin admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:22.515215 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfd7ce1-daf7-4f69-bc3f-87f0e097c87e admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:22.515496 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfd7ce1-daf7-4f69-bc3f-87f0e097c87e admin admin] PATH_INFO: `/v3/projects/4103c823578e4fc0a5a96b7a4e631855/users/a04507052803403ba5a5f29b416220fa/roles/9baff2b18f2649f7b757ff2e1f63c853` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:22.517597 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfd7ce1-daf7-4f69-bc3f-87f0e097c87e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4103c823578e4fc0a5a96b7a4e631855, user_id=a04507052803403ba5a5f29b416220fa, role_id=9baff2b18f2649f7b757ff2e1f63c853)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:22.520340 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfd7ce1-daf7-4f69-bc3f-87f0e097c87e admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:22.532354 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1001/1979] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:40:22 2022] PUT /identity/v3/projects/4103c823578e4fc0a5a96b7a4e631855/users/a04507052803403ba5a5f29b416220fa/roles/9baff2b18f2649f7b757ff2e1f63c853 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:40:22.543381 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e218a1f1-b518-468a-a748-fb3eac0e9bb8 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:22.543673 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e218a1f1-b518-468a-a748-fb3eac0e9bb8 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:22.543955 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e218a1f1-b518-468a-a748-fb3eac0e9bb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:22.557521 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 979/1980] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:40:22 2022] DELETE /identity/v3/users/d95a768ee823408dad5d9c831201a5fc => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:40:22.575317 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-e218a1f1-b518-468a-a748-fb3eac0e9bb8 None None] MFA Rules not processed for user `a04507052803403ba5a5f29b416220fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:40:22.577830 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-e218a1f1-b518-468a-a748-fb3eac0e9bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:22.647803 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:40:22.658600 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1002/1981] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:40:22 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:40:22.669387 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b263f5a-68b8-4728-baa9-747785b8ceb7 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:22.671113 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-6b263f5a-68b8-4728-baa9-747785b8ceb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:22.773912 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b263f5a-68b8-4728-baa9-747785b8ceb7 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:22.781422 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53632da0-ce97-4f34-9f82-7092bb5a511d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d81f597cb44c56a966537089bcce4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:22.782540 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53632da0-ce97-4f34-9f82-7092bb5a511d service cinder] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:22.783023 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53632da0-ce97-4f34-9f82-7092bb5a511d service cinder] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:22.783494 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53632da0-ce97-4f34-9f82-7092bb5a511d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:22.811460 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53632da0-ce97-4f34-9f82-7092bb5a511d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:22.815744 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53632da0-ce97-4f34-9f82-7092bb5a511d service cinder] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:22.830710 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 980/1982] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:40:22 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:40:23.022330 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e218a1f1-b518-468a-a748-fb3eac0e9bb8 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:23.023744 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-e218a1f1-b518-468a-a748-fb3eac0e9bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:23.121877 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e218a1f1-b518-468a-a748-fb3eac0e9bb8 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:23.127243 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e89b745-194c-4c1a-ab97-e4aa227d1009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:23.128101 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e89b745-194c-4c1a-ab97-e4aa227d1009 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:23.128436 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e89b745-194c-4c1a-ab97-e4aa227d1009 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:23.128724 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e89b745-194c-4c1a-ab97-e4aa227d1009 admin admin] PATH_INFO: `/v3/projects/6bfb0c1a212640c096e4c08cf830464e` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:23.129951 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e89b745-194c-4c1a-ab97-e4aa227d1009 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bfb0c1a212640c096e4c08cf830464e)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:23.131842 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e89b745-194c-4c1a-ab97-e4aa227d1009 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:23.221532 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53632da0-ce97-4f34-9f82-7092bb5a511d service cinder] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:23.230369 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53632da0-ce97-4f34-9f82-7092bb5a511d service cinder] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:23.234950 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae582da-68e8-4473-9e75-8a34cffe4091 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d81f597cb44c56a966537089bcce4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:23.235638 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae582da-68e8-4473-9e75-8a34cffe4091 service cinder] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:23.235928 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae582da-68e8-4473-9e75-8a34cffe4091 service cinder] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:23.236209 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae582da-68e8-4473-9e75-8a34cffe4091 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:23.238055 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-3ae582da-68e8-4473-9e75-8a34cffe4091 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:23.268672 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-8e89b745-194c-4c1a-ab97-e4aa227d1009 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bfb0c1a212640c096e4c08cf830464e', 'request_id': 'req-8e89b745-194c-4c1a-ab97-e4aa227d1009'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:23.287562 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:40:23.309293 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae582da-68e8-4473-9e75-8a34cffe4091 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:23.310139 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae582da-68e8-4473-9e75-8a34cffe4091 service cinder] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:23.322648 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 981/1983] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:40:23 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 14:40:23.350984 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-8e89b745-194c-4c1a-ab97-e4aa227d1009 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bfb0c1a212640c096e4c08cf830464e', 'request_id': 'req-8e89b745-194c-4c1a-ab97-e4aa227d1009'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:23.360441 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-8e89b745-194c-4c1a-ab97-e4aa227d1009 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bfb0c1a212640c096e4c08cf830464e', 'request_id': 'req-8e89b745-194c-4c1a-ab97-e4aa227d1009'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:23.374832 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-8e89b745-194c-4c1a-ab97-e4aa227d1009 admin admin] The token cache is being invalidate because project 6bfb0c1a212640c096e4c08cf830464e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:40:23.375197 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-8e89b745-194c-4c1a-ab97-e4aa227d1009 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:23.375950 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-8e89b745-194c-4c1a-ab97-e4aa227d1009 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:23.385436 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1003/1984] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:40:23 2022] DELETE /identity/v3/projects/6bfb0c1a212640c096e4c08cf830464e => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:40:23.387102 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae582da-68e8-4473-9e75-8a34cffe4091 service cinder] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:23.400390 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae582da-68e8-4473-9e75-8a34cffe4091 service cinder] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:23.471087 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1839f62f-6b98-4e4d-a30e-6047e479313d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:23.471997 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1839f62f-6b98-4e4d-a30e-6047e479313d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:23.472354 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1839f62f-6b98-4e4d-a30e-6047e479313d admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:23.472669 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1839f62f-6b98-4e4d-a30e-6047e479313d admin admin] PATH_INFO: `/v3/users/ef6bb1ac3ec04e44963d12ad523a70b9` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:23.473918 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1839f62f-6b98-4e4d-a30e-6047e479313d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef6bb1ac3ec04e44963d12ad523a70b9)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:23.475904 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1839f62f-6b98-4e4d-a30e-6047e479313d admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:23.724577 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-1839f62f-6b98-4e4d-a30e-6047e479313d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef6bb1ac3ec04e44963d12ad523a70b9'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:23.725153 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-1839f62f-6b98-4e4d-a30e-6047e479313d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef6bb1ac3ec04e44963d12ad523a70b9'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:23.725774 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-1839f62f-6b98-4e4d-a30e-6047e479313d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef6bb1ac3ec04e44963d12ad523a70b9'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:23.732946 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-1839f62f-6b98-4e4d-a30e-6047e479313d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef6bb1ac3ec04e44963d12ad523a70b9'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:23.746081 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-1839f62f-6b98-4e4d-a30e-6047e479313d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef6bb1ac3ec04e44963d12ad523a70b9'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:23.796604 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 982/1985] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:40:23 2022] DELETE /identity/v3/users/ef6bb1ac3ec04e44963d12ad523a70b9 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:40:24.198786 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e89b745-194c-4c1a-ab97-e4aa227d1009 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:24.200395 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-8e89b745-194c-4c1a-ab97-e4aa227d1009 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:24.214036 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e89b745-194c-4c1a-ab97-e4aa227d1009 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:24.282424 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa27077d-79aa-4f1f-813f-0ccf92d124e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:24.283327 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27077d-79aa-4f1f-813f-0ccf92d124e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:24.283608 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27077d-79aa-4f1f-813f-0ccf92d124e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:24.283882 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27077d-79aa-4f1f-813f-0ccf92d124e3 admin admin] PATH_INFO: `/v3/projects/08dd2d916fab436484b50a2912f0e946` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:24.285140 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa27077d-79aa-4f1f-813f-0ccf92d124e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08dd2d916fab436484b50a2912f0e946)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:24.287056 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa27077d-79aa-4f1f-813f-0ccf92d124e3 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:24.394214 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:40:24.412637 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-fa27077d-79aa-4f1f-813f-0ccf92d124e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08dd2d916fab436484b50a2912f0e946', 'request_id': 'req-fa27077d-79aa-4f1f-813f-0ccf92d124e3'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:24.447550 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-fa27077d-79aa-4f1f-813f-0ccf92d124e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08dd2d916fab436484b50a2912f0e946', 'request_id': 'req-fa27077d-79aa-4f1f-813f-0ccf92d124e3'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:24.452965 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-fa27077d-79aa-4f1f-813f-0ccf92d124e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08dd2d916fab436484b50a2912f0e946', 'request_id': 'req-fa27077d-79aa-4f1f-813f-0ccf92d124e3'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:24.464030 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-fa27077d-79aa-4f1f-813f-0ccf92d124e3 admin admin] The token cache is being invalidate because project 08dd2d916fab436484b50a2912f0e946 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:40:24.464316 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-fa27077d-79aa-4f1f-813f-0ccf92d124e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:24.464958 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-fa27077d-79aa-4f1f-813f-0ccf92d124e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:24.466501 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1004/1986] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:40:24 2022] DELETE /identity/v3/projects/08dd2d916fab436484b50a2912f0e946 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:40:24.473122 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc7a44b-45b5-4472-988f-0b2971998a13 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:24.473632 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc7a44b-45b5-4472-988f-0b2971998a13 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:24.473725 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc7a44b-45b5-4472-988f-0b2971998a13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:24.492073 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-1bc7a44b-45b5-4472-988f-0b2971998a13 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:40:24.493264 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-1bc7a44b-45b5-4472-988f-0b2971998a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:24.551799 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 983/1987] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:40:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:40:24.560150 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3dc01e-cce2-46a5-9e3a-f534cbe4b804 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:24.560484 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3dc01e-cce2-46a5-9e3a-f534cbe4b804 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:24.560780 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3dc01e-cce2-46a5-9e3a-f534cbe4b804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:24.579473 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-6b3dc01e-cce2-46a5-9e3a-f534cbe4b804 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:40:24.580747 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-6b3dc01e-cce2-46a5-9e3a-f534cbe4b804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:24.589031 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1005/1988] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:40:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:40:24.593871 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc7a44b-45b5-4472-988f-0b2971998a13 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:24.598822 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:40:24.606792 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc7a44b-45b5-4472-988f-0b2971998a13 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:24.614647 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464f6fd3-087a-411e-936c-985462ea14e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:24.615363 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f6fd3-087a-411e-936c-985462ea14e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:24.615652 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f6fd3-087a-411e-936c-985462ea14e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:24.615933 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f6fd3-087a-411e-936c-985462ea14e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:24.616722 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464f6fd3-087a-411e-936c-985462ea14e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:40:24.617035 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464f6fd3-087a-411e-936c-985462ea14e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:24.618721 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464f6fd3-087a-411e-936c-985462ea14e9 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:24.631508 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 984/1989] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jun 7 14:40:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:40:24.635962 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3dc01e-cce2-46a5-9e3a-f534cbe4b804 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:24.646088 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3dc01e-cce2-46a5-9e3a-f534cbe4b804 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:24.650801 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ef589e-de34-402e-ae69-5c858cfcc04c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:24.651477 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ef589e-de34-402e-ae69-5c858cfcc04c admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:24.651765 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ef589e-de34-402e-ae69-5c858cfcc04c admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:24.652044 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ef589e-de34-402e-ae69-5c858cfcc04c admin admin] PATH_INFO: `/v3/projects` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:24.652786 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ef589e-de34-402e-ae69-5c858cfcc04c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:24.655334 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ef589e-de34-402e-ae69-5c858cfcc04c admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:24.787791 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1006/1990] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:40:24 2022] POST /identity/v3/projects => generated 373 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:40:24.791921 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464f6fd3-087a-411e-936c-985462ea14e9 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:24.802064 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464f6fd3-087a-411e-936c-985462ea14e9 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:24.857742 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9354eccb-6862-448e-94d5-d06551a1258b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:24.858413 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9354eccb-6862-448e-94d5-d06551a1258b admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:24.858699 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9354eccb-6862-448e-94d5-d06551a1258b admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:24.858975 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9354eccb-6862-448e-94d5-d06551a1258b admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:24.859962 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9354eccb-6862-448e-94d5-d06551a1258b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:24.861563 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9354eccb-6862-448e-94d5-d06551a1258b admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:24.929004 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 985/1991] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:40:24 2022] POST /identity/v3/users => generated 287 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:40:24.933260 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ef589e-de34-402e-ae69-5c858cfcc04c admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:24.944483 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ef589e-de34-402e-ae69-5c858cfcc04c admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:24.949206 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad14808b-a84c-460f-89cc-87a7e6c0e0bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:24.949905 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad14808b-a84c-460f-89cc-87a7e6c0e0bf admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:24.950201 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad14808b-a84c-460f-89cc-87a7e6c0e0bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:24.950477 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad14808b-a84c-460f-89cc-87a7e6c0e0bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:24.951237 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad14808b-a84c-460f-89cc-87a7e6c0e0bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:24.953532 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad14808b-a84c-460f-89cc-87a7e6c0e0bf admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:24.964634 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1007/1992] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:40:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:40:24.968957 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9354eccb-6862-448e-94d5-d06551a1258b admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:24.979129 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9354eccb-6862-448e-94d5-d06551a1258b admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:24.984821 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98304c6a-5def-49b6-9a39-7537eff62c05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:24.985494 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98304c6a-5def-49b6-9a39-7537eff62c05 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:24.986239 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98304c6a-5def-49b6-9a39-7537eff62c05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:24.986239 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98304c6a-5def-49b6-9a39-7537eff62c05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:24.986907 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98304c6a-5def-49b6-9a39-7537eff62c05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:24.988564 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98304c6a-5def-49b6-9a39-7537eff62c05 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:25.001948 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 986/1993] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:40:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:40:25.003777 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad14808b-a84c-460f-89cc-87a7e6c0e0bf admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:25.015005 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad14808b-a84c-460f-89cc-87a7e6c0e0bf admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:25.019973 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdfad14-8a76-4af4-ab95-59a37f2a94af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:25.021012 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdfad14-8a76-4af4-ab95-59a37f2a94af admin admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:25.021398 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdfad14-8a76-4af4-ab95-59a37f2a94af admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:25.021751 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdfad14-8a76-4af4-ab95-59a37f2a94af admin admin] PATH_INFO: `/v3/projects/40a2691996124d37a5a292f7ea4a2f78/users/122c179007b443439f84a0fe28868214/roles/3771341b3be641139e752aab690397a4` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:25.037464 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdfad14-8a76-4af4-ab95-59a37f2a94af admin admin] RBAC: Authorizing `identity:create_grant(project_id=40a2691996124d37a5a292f7ea4a2f78, user_id=122c179007b443439f84a0fe28868214, role_id=3771341b3be641139e752aab690397a4)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:25.040293 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdfad14-8a76-4af4-ab95-59a37f2a94af admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:25.117100 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1008/1994] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:40:25 2022] PUT /identity/v3/projects/40a2691996124d37a5a292f7ea4a2f78/users/122c179007b443439f84a0fe28868214/roles/3771341b3be641139e752aab690397a4 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:40:25.129489 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5518af16-2a89-4e12-8bc9-0a9356cfbee0 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:25.129782 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5518af16-2a89-4e12-8bc9-0a9356cfbee0 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:25.130070 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5518af16-2a89-4e12-8bc9-0a9356cfbee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:25.158005 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-5518af16-2a89-4e12-8bc9-0a9356cfbee0 None None] MFA Rules not processed for user `122c179007b443439f84a0fe28868214`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:40:25.159218 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-5518af16-2a89-4e12-8bc9-0a9356cfbee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:25.223338 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 987/1995] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:40:25 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:40:25.235103 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdfad14-8a76-4af4-ab95-59a37f2a94af admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:25.236804 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-2bdfad14-8a76-4af4-ab95-59a37f2a94af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:25.294141 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdfad14-8a76-4af4-ab95-59a37f2a94af admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:25.298475 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5041245b-9bf6-48da-9ff0-c7cee99f6bb6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51da9867d9147f198b8077e953c4f9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:25.299131 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041245b-9bf6-48da-9ff0-c7cee99f6bb6 service glance] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:25.299432 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041245b-9bf6-48da-9ff0-c7cee99f6bb6 service glance] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:25.299826 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041245b-9bf6-48da-9ff0-c7cee99f6bb6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:25.309007 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5041245b-9bf6-48da-9ff0-c7cee99f6bb6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:25.310271 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5041245b-9bf6-48da-9ff0-c7cee99f6bb6 service glance] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:25.320175 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1009/1996] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:40:25 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:40:35.161437 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5518af16-2a89-4e12-8bc9-0a9356cfbee0 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:35.168196 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-5518af16-2a89-4e12-8bc9-0a9356cfbee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:35.230601 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:40:35.235220 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5518af16-2a89-4e12-8bc9-0a9356cfbee0 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:35.243206 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259c76ab-fab1-4c1e-930d-f1feab1685ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:35.244121 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259c76ab-fab1-4c1e-930d-f1feab1685ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:35.244433 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259c76ab-fab1-4c1e-930d-f1feab1685ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:35.244716 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259c76ab-fab1-4c1e-930d-f1feab1685ea admin admin] PATH_INFO: `/v3/users/a72376ecb8dc425e821b06b24bd6b9e3` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:35.245956 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259c76ab-fab1-4c1e-930d-f1feab1685ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=a72376ecb8dc425e821b06b24bd6b9e3)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:35.247960 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259c76ab-fab1-4c1e-930d-f1feab1685ea admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:35.315905 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=67,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:40:35.438714 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-259c76ab-fab1-4c1e-930d-f1feab1685ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a72376ecb8dc425e821b06b24bd6b9e3'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:35.439372 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-259c76ab-fab1-4c1e-930d-f1feab1685ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a72376ecb8dc425e821b06b24bd6b9e3'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:35.439972 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-259c76ab-fab1-4c1e-930d-f1feab1685ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a72376ecb8dc425e821b06b24bd6b9e3'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:35.446924 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-259c76ab-fab1-4c1e-930d-f1feab1685ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a72376ecb8dc425e821b06b24bd6b9e3'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:35.458771 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-259c76ab-fab1-4c1e-930d-f1feab1685ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a72376ecb8dc425e821b06b24bd6b9e3'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:35.527587 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 988/1997] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:40:35 2022] DELETE /identity/v3/users/a72376ecb8dc425e821b06b24bd6b9e3 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:40:35.532159 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ce1c6-7181-4bde-b693-af95795c74c0 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:35.532495 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ce1c6-7181-4bde-b693-af95795c74c0 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:35.533274 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ce1c6-7181-4bde-b693-af95795c74c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:35.551285 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-ff1ce1c6-7181-4bde-b693-af95795c74c0 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:40:35.552531 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-ff1ce1c6-7181-4bde-b693-af95795c74c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:35.612279 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1010/1998] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:40:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:40:35.621840 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259c76ab-fab1-4c1e-930d-f1feab1685ea admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:35.623243 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-259c76ab-fab1-4c1e-930d-f1feab1685ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:35.683353 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259c76ab-fab1-4c1e-930d-f1feab1685ea admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:35.687034 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02db269-9a16-4898-99e4-55b2f0d0536c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:35.687694 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02db269-9a16-4898-99e4-55b2f0d0536c service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:35.687984 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02db269-9a16-4898-99e4-55b2f0d0536c service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:35.688269 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02db269-9a16-4898-99e4-55b2f0d0536c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:35.698021 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02db269-9a16-4898-99e4-55b2f0d0536c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:35.699445 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02db269-9a16-4898-99e4-55b2f0d0536c service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:35.709786 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 989/1999] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:40:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:40:36.141707 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1ce1c6-7181-4bde-b693-af95795c74c0 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:36.143793 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-ff1ce1c6-7181-4bde-b693-af95795c74c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:36.156136 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1ce1c6-7181-4bde-b693-af95795c74c0 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:36.160916 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8478019-dc82-474b-952b-8869892de634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:36.161783 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8478019-dc82-474b-952b-8869892de634 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:36.162069 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8478019-dc82-474b-952b-8869892de634 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:36.162348 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8478019-dc82-474b-952b-8869892de634 admin admin] PATH_INFO: `/v3/projects/6bed7945f0ae41a6a2330a1946ad5be9` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:36.163547 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8478019-dc82-474b-952b-8869892de634 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bed7945f0ae41a6a2330a1946ad5be9)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:36.165511 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8478019-dc82-474b-952b-8869892de634 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:36.270768 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-b8478019-dc82-474b-952b-8869892de634 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bed7945f0ae41a6a2330a1946ad5be9', 'request_id': 'req-b8478019-dc82-474b-952b-8869892de634'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:36.322638 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-b8478019-dc82-474b-952b-8869892de634 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bed7945f0ae41a6a2330a1946ad5be9', 'request_id': 'req-b8478019-dc82-474b-952b-8869892de634'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:36.332312 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-b8478019-dc82-474b-952b-8869892de634 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bed7945f0ae41a6a2330a1946ad5be9', 'request_id': 'req-b8478019-dc82-474b-952b-8869892de634'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:36.347270 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-b8478019-dc82-474b-952b-8869892de634 admin admin] The token cache is being invalidate because project 6bed7945f0ae41a6a2330a1946ad5be9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:40:36.347575 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-b8478019-dc82-474b-952b-8869892de634 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:36.348552 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-b8478019-dc82-474b-952b-8869892de634 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:36.349783 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1011/2000] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:40:36 2022] DELETE /identity/v3/projects/6bed7945f0ae41a6a2330a1946ad5be9 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:40:36.354709 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02db269-9a16-4898-99e4-55b2f0d0536c service neutron] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:36.365076 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02db269-9a16-4898-99e4-55b2f0d0536c service neutron] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:36.423426 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069f175c-9902-4d5c-b5be-d793c69ec7be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:36.424875 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069f175c-9902-4d5c-b5be-d793c69ec7be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:36.425165 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069f175c-9902-4d5c-b5be-d793c69ec7be admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:36.425446 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069f175c-9902-4d5c-b5be-d793c69ec7be admin admin] PATH_INFO: `/v3/users/3e8b3032ec5f45c38ba6d7f246298ab2` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:36.427020 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069f175c-9902-4d5c-b5be-d793c69ec7be admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e8b3032ec5f45c38ba6d7f246298ab2)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:36.429024 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069f175c-9902-4d5c-b5be-d793c69ec7be admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:36.452282 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:40:36.690562 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-069f175c-9902-4d5c-b5be-d793c69ec7be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e8b3032ec5f45c38ba6d7f246298ab2'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:36.691129 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-069f175c-9902-4d5c-b5be-d793c69ec7be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e8b3032ec5f45c38ba6d7f246298ab2'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:36.691723 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-069f175c-9902-4d5c-b5be-d793c69ec7be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e8b3032ec5f45c38ba6d7f246298ab2'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:36.698633 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-069f175c-9902-4d5c-b5be-d793c69ec7be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e8b3032ec5f45c38ba6d7f246298ab2'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:36.711298 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-069f175c-9902-4d5c-b5be-d793c69ec7be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e8b3032ec5f45c38ba6d7f246298ab2'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:36.762274 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 990/2001] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:40:36 2022] DELETE /identity/v3/users/3e8b3032ec5f45c38ba6d7f246298ab2 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:40:37.169567 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8478019-dc82-474b-952b-8869892de634 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:37.171159 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-b8478019-dc82-474b-952b-8869892de634 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:37.186727 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8478019-dc82-474b-952b-8869892de634 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:37.254272 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa9d169-f7ae-496e-ae8d-621a105b6fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:37.255153 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa9d169-f7ae-496e-ae8d-621a105b6fc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:37.255454 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa9d169-f7ae-496e-ae8d-621a105b6fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:37.255739 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa9d169-f7ae-496e-ae8d-621a105b6fc3 admin admin] PATH_INFO: `/v3/projects/0a052779b5ab475f958b7baf183d22f8` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:37.257430 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa9d169-f7ae-496e-ae8d-621a105b6fc3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a052779b5ab475f958b7baf183d22f8)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:37.258906 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa9d169-f7ae-496e-ae8d-621a105b6fc3 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:37.365193 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:40:37.385199 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-0aa9d169-f7ae-496e-ae8d-621a105b6fc3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a052779b5ab475f958b7baf183d22f8', 'request_id': 'req-0aa9d169-f7ae-496e-ae8d-621a105b6fc3'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:37.465038 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-0aa9d169-f7ae-496e-ae8d-621a105b6fc3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a052779b5ab475f958b7baf183d22f8', 'request_id': 'req-0aa9d169-f7ae-496e-ae8d-621a105b6fc3'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:37.472262 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-0aa9d169-f7ae-496e-ae8d-621a105b6fc3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a052779b5ab475f958b7baf183d22f8', 'request_id': 'req-0aa9d169-f7ae-496e-ae8d-621a105b6fc3'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:37.483845 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-0aa9d169-f7ae-496e-ae8d-621a105b6fc3 admin admin] The token cache is being invalidate because project 0a052779b5ab475f958b7baf183d22f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:40:37.484040 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-0aa9d169-f7ae-496e-ae8d-621a105b6fc3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:37.484850 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-0aa9d169-f7ae-496e-ae8d-621a105b6fc3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:37.486417 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1012/2002] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:40:37 2022] DELETE /identity/v3/projects/0a052779b5ab475f958b7baf183d22f8 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:40:37.491680 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444e020d-59cb-4ce5-95b9-4633d1737cfb None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:37.492212 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444e020d-59cb-4ce5-95b9-4633d1737cfb None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:37.492343 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444e020d-59cb-4ce5-95b9-4633d1737cfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:37.514523 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-444e020d-59cb-4ce5-95b9-4633d1737cfb None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:40:37.515766 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-444e020d-59cb-4ce5-95b9-4633d1737cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:37.595074 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 991/2003] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:40:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:40:37.603590 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77c25c0-0b51-4a76-97bc-074743e7e2e3 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:37.604071 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77c25c0-0b51-4a76-97bc-074743e7e2e3 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:37.604398 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77c25c0-0b51-4a76-97bc-074743e7e2e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:37.630364 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-d77c25c0-0b51-4a76-97bc-074743e7e2e3 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:40:37.631565 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-d77c25c0-0b51-4a76-97bc-074743e7e2e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:37.640080 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1013/2004] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:40:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:40:37.645559 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444e020d-59cb-4ce5-95b9-4633d1737cfb None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:37.656764 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444e020d-59cb-4ce5-95b9-4633d1737cfb None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:37.662497 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d3dee9-102b-456a-b150-bdb4ac27e595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:37.663789 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d3dee9-102b-456a-b150-bdb4ac27e595 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:37.664552 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d3dee9-102b-456a-b150-bdb4ac27e595 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:37.665085 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d3dee9-102b-456a-b150-bdb4ac27e595 admin admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:37.666612 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d3dee9-102b-456a-b150-bdb4ac27e595 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:40:37.666959 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d3dee9-102b-456a-b150-bdb4ac27e595 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:37.670528 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d3dee9-102b-456a-b150-bdb4ac27e595 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:37.697686 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 992/2005] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jun 7 14:40:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:40:37.704030 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77c25c0-0b51-4a76-97bc-074743e7e2e3 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:37.718921 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77c25c0-0b51-4a76-97bc-074743e7e2e3 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:37.724234 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b17054-1fa0-44ac-bed7-b15563dc3d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:37.724939 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b17054-1fa0-44ac-bed7-b15563dc3d37 admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:37.725250 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b17054-1fa0-44ac-bed7-b15563dc3d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:37.725498 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b17054-1fa0-44ac-bed7-b15563dc3d37 admin admin] PATH_INFO: `/v3/projects` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:37.726276 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b17054-1fa0-44ac-bed7-b15563dc3d37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:37.728404 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b17054-1fa0-44ac-bed7-b15563dc3d37 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:37.827468 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1014/2006] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:40:37 2022] POST /identity/v3/projects => generated 389 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:40:37.832238 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d3dee9-102b-456a-b150-bdb4ac27e595 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:37.842453 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d3dee9-102b-456a-b150-bdb4ac27e595 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:37.915317 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8745fe0b-851e-4864-b588-f88a1c976be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:37.916000 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8745fe0b-851e-4864-b588-f88a1c976be3 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:37.916291 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8745fe0b-851e-4864-b588-f88a1c976be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:37.916606 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8745fe0b-851e-4864-b588-f88a1c976be3 admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:37.917316 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8745fe0b-851e-4864-b588-f88a1c976be3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:37.919275 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8745fe0b-851e-4864-b588-f88a1c976be3 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:37.992210 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 993/2007] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:40:37 2022] POST /identity/v3/users => generated 295 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:40:37.997038 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b17054-1fa0-44ac-bed7-b15563dc3d37 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:38.008159 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b17054-1fa0-44ac-bed7-b15563dc3d37 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:38.013248 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7506e7fe-2131-423b-864b-fc5f9dc5bb94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:38.013960 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7506e7fe-2131-423b-864b-fc5f9dc5bb94 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:38.014253 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7506e7fe-2131-423b-864b-fc5f9dc5bb94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:38.014531 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7506e7fe-2131-423b-864b-fc5f9dc5bb94 admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:38.015265 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7506e7fe-2131-423b-864b-fc5f9dc5bb94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:38.017758 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7506e7fe-2131-423b-864b-fc5f9dc5bb94 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:38.028335 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1015/2008] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:40:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:40:38.032609 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8745fe0b-851e-4864-b588-f88a1c976be3 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:38.043190 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8745fe0b-851e-4864-b588-f88a1c976be3 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:38.048303 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95d6e93-0e76-4078-9abb-256eef48659d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:38.049537 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95d6e93-0e76-4078-9abb-256eef48659d admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:38.049825 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95d6e93-0e76-4078-9abb-256eef48659d admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:38.050102 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95d6e93-0e76-4078-9abb-256eef48659d admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:38.050833 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95d6e93-0e76-4078-9abb-256eef48659d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:38.052745 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95d6e93-0e76-4078-9abb-256eef48659d admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:38.063410 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 994/2009] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:40:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:40:38.068602 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7506e7fe-2131-423b-864b-fc5f9dc5bb94 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:38.080920 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7506e7fe-2131-423b-864b-fc5f9dc5bb94 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:38.085822 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac7a967-9ee0-4b4c-a406-f136b8e6d0c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:38.086726 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac7a967-9ee0-4b4c-a406-f136b8e6d0c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:38.087071 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac7a967-9ee0-4b4c-a406-f136b8e6d0c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:38.087294 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac7a967-9ee0-4b4c-a406-f136b8e6d0c9 admin admin] PATH_INFO: `/v3/projects/ad0b61f7690f45118ecd28f7dbc7feea/users/4871205670f640499040fd5d9fdda3d9/roles/3771341b3be641139e752aab690397a4` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:38.103743 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac7a967-9ee0-4b4c-a406-f136b8e6d0c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad0b61f7690f45118ecd28f7dbc7feea, user_id=4871205670f640499040fd5d9fdda3d9, role_id=3771341b3be641139e752aab690397a4)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:38.106416 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac7a967-9ee0-4b4c-a406-f136b8e6d0c9 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:38.215858 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1016/2010] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:40:38 2022] PUT /identity/v3/projects/ad0b61f7690f45118ecd28f7dbc7feea/users/4871205670f640499040fd5d9fdda3d9/roles/3771341b3be641139e752aab690397a4 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:40:38.226500 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3354be93-5020-4efc-8222-df01c04d5156 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:38.226799 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3354be93-5020-4efc-8222-df01c04d5156 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:38.227144 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3354be93-5020-4efc-8222-df01c04d5156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:38.235462 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:40:38.256543 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-3354be93-5020-4efc-8222-df01c04d5156 None None] MFA Rules not processed for user `4871205670f640499040fd5d9fdda3d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:40:38.257826 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-3354be93-5020-4efc-8222-df01c04d5156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:38.328967 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 995/2011] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:40:38 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:40:38.333671 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac7a967-9ee0-4b4c-a406-f136b8e6d0c9 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:38.399850 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac7a967-9ee0-4b4c-a406-f136b8e6d0c9 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:38.404732 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca9fa26-e9b8-484e-83d2-9e24f2f39623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:38.405395 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca9fa26-e9b8-484e-83d2-9e24f2f39623 admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:38.405680 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca9fa26-e9b8-484e-83d2-9e24f2f39623 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:38.406204 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca9fa26-e9b8-484e-83d2-9e24f2f39623 admin admin] PATH_INFO: `/v3/projects` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:38.406642 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca9fa26-e9b8-484e-83d2-9e24f2f39623 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:38.409083 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca9fa26-e9b8-484e-83d2-9e24f2f39623 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:38.518616 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1017/2012] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:40:38 2022] POST /identity/v3/projects => generated 389 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:40:38.522869 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3354be93-5020-4efc-8222-df01c04d5156 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:38.534413 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3354be93-5020-4efc-8222-df01c04d5156 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:38.588703 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0a7a6b-4448-4dc5-b7d0-6a203eda3966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:38.589318 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0a7a6b-4448-4dc5-b7d0-6a203eda3966 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:38.589686 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0a7a6b-4448-4dc5-b7d0-6a203eda3966 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:38.589982 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0a7a6b-4448-4dc5-b7d0-6a203eda3966 admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:38.590726 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0a7a6b-4448-4dc5-b7d0-6a203eda3966 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:38.592649 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0a7a6b-4448-4dc5-b7d0-6a203eda3966 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:38.671812 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 996/2013] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:40:38 2022] POST /identity/v3/users => generated 301 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:40:38.675870 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca9fa26-e9b8-484e-83d2-9e24f2f39623 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:38.686090 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca9fa26-e9b8-484e-83d2-9e24f2f39623 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:38.691208 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf74885-1f45-4165-aa83-e13bb72c9f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:38.691897 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf74885-1f45-4165-aa83-e13bb72c9f0e admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:38.692193 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf74885-1f45-4165-aa83-e13bb72c9f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:38.693517 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf74885-1f45-4165-aa83-e13bb72c9f0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:38.694634 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf74885-1f45-4165-aa83-e13bb72c9f0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:38.696446 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf74885-1f45-4165-aa83-e13bb72c9f0e admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:38.710446 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1018/2014] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:40:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:40:38.714600 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0a7a6b-4448-4dc5-b7d0-6a203eda3966 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:38.725323 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0a7a6b-4448-4dc5-b7d0-6a203eda3966 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:38.730381 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b78ceb3-a5f6-4305-a352-788de628d33e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:38.740520 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b78ceb3-a5f6-4305-a352-788de628d33e admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:38.740520 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b78ceb3-a5f6-4305-a352-788de628d33e admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:38.740520 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b78ceb3-a5f6-4305-a352-788de628d33e admin admin] PATH_INFO: `/v3/projects/c5b9e8af81dc49a19275553b57e916fd/users/6636a875310e4d869860a805c0397275/roles/9baff2b18f2649f7b757ff2e1f63c853` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:38.750282 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b78ceb3-a5f6-4305-a352-788de628d33e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5b9e8af81dc49a19275553b57e916fd, user_id=6636a875310e4d869860a805c0397275, role_id=9baff2b18f2649f7b757ff2e1f63c853)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:38.752981 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b78ceb3-a5f6-4305-a352-788de628d33e admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:38.854295 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 997/2015] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:40:38 2022] PUT /identity/v3/projects/c5b9e8af81dc49a19275553b57e916fd/users/6636a875310e4d869860a805c0397275/roles/9baff2b18f2649f7b757ff2e1f63c853 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:40:38.858169 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf74885-1f45-4165-aa83-e13bb72c9f0e admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:38.868211 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf74885-1f45-4165-aa83-e13bb72c9f0e admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:38.887739 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:40:38.928465 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2448e21f-0db9-4b29-866e-eefacd6ee558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:38.929175 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2448e21f-0db9-4b29-866e-eefacd6ee558 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:38.929488 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2448e21f-0db9-4b29-866e-eefacd6ee558 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:38.929882 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2448e21f-0db9-4b29-866e-eefacd6ee558 admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:38.930624 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2448e21f-0db9-4b29-866e-eefacd6ee558 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:38.932223 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2448e21f-0db9-4b29-866e-eefacd6ee558 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:38.942736 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1019/2016] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:40:38 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:40:38.946169 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b78ceb3-a5f6-4305-a352-788de628d33e admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:38.957148 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b78ceb3-a5f6-4305-a352-788de628d33e admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:38.962680 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0509dca-fd3f-4276-bfbb-f99f4a938502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:38.963587 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0509dca-fd3f-4276-bfbb-f99f4a938502 admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:38.963876 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0509dca-fd3f-4276-bfbb-f99f4a938502 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:38.964157 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0509dca-fd3f-4276-bfbb-f99f4a938502 admin admin] PATH_INFO: `/v3/projects/c5b9e8af81dc49a19275553b57e916fd/users/6636a875310e4d869860a805c0397275/roles/9baff2b18f2649f7b757ff2e1f63c853` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:38.966474 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0509dca-fd3f-4276-bfbb-f99f4a938502 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5b9e8af81dc49a19275553b57e916fd, user_id=6636a875310e4d869860a805c0397275, role_id=9baff2b18f2649f7b757ff2e1f63c853)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:38.969159 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0509dca-fd3f-4276-bfbb-f99f4a938502 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:38.980633 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 998/2017] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:40:38 2022] PUT /identity/v3/projects/c5b9e8af81dc49a19275553b57e916fd/users/6636a875310e4d869860a805c0397275/roles/9baff2b18f2649f7b757ff2e1f63c853 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:40:38.991453 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fda0b5-e87e-4cb8-bd52-9299df47c3e7 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:38.991751 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fda0b5-e87e-4cb8-bd52-9299df47c3e7 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:38.992039 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fda0b5-e87e-4cb8-bd52-9299df47c3e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:39.018599 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-90fda0b5-e87e-4cb8-bd52-9299df47c3e7 None None] MFA Rules not processed for user `6636a875310e4d869860a805c0397275`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:40:39.019809 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-90fda0b5-e87e-4cb8-bd52-9299df47c3e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:39.091397 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1020/2018] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:40:38 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:40:39.103453 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0509dca-fd3f-4276-bfbb-f99f4a938502 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:39.105215 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-b0509dca-fd3f-4276-bfbb-f99f4a938502 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:39.158302 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:40:39.164087 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0509dca-fd3f-4276-bfbb-f99f4a938502 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:39.167641 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c060eaae-ed68-49e4-ac82-d5fb7250736d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d81f597cb44c56a966537089bcce4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:39.168471 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c060eaae-ed68-49e4-ac82-d5fb7250736d service cinder] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:39.171625 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c060eaae-ed68-49e4-ac82-d5fb7250736d service cinder] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:39.171625 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c060eaae-ed68-49e4-ac82-d5fb7250736d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:39.180163 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c060eaae-ed68-49e4-ac82-d5fb7250736d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:39.181331 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c060eaae-ed68-49e4-ac82-d5fb7250736d service cinder] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:39.193025 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 999/2019] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:40:39 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:40:39.327710 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fda0b5-e87e-4cb8-bd52-9299df47c3e7 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:39.389673 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fda0b5-e87e-4cb8-bd52-9299df47c3e7 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:39.394733 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e897d2eb-d96f-4fdc-b566-0b33c716a417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:39.395600 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e897d2eb-d96f-4fdc-b566-0b33c716a417 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:39.395875 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e897d2eb-d96f-4fdc-b566-0b33c716a417 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:39.396145 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e897d2eb-d96f-4fdc-b566-0b33c716a417 admin admin] PATH_INFO: `/v3/users/4871205670f640499040fd5d9fdda3d9` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:39.397459 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e897d2eb-d96f-4fdc-b566-0b33c716a417 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4871205670f640499040fd5d9fdda3d9)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:39.399382 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e897d2eb-d96f-4fdc-b566-0b33c716a417 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:39.684160 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-e897d2eb-d96f-4fdc-b566-0b33c716a417 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4871205670f640499040fd5d9fdda3d9'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:39.685149 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-e897d2eb-d96f-4fdc-b566-0b33c716a417 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4871205670f640499040fd5d9fdda3d9'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:39.685516 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-e897d2eb-d96f-4fdc-b566-0b33c716a417 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4871205670f640499040fd5d9fdda3d9'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:39.686929 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:40:39.694661 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-e897d2eb-d96f-4fdc-b566-0b33c716a417 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4871205670f640499040fd5d9fdda3d9'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:39.715273 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-e897d2eb-d96f-4fdc-b566-0b33c716a417 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4871205670f640499040fd5d9fdda3d9'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:39.832937 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1021/2020] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:40:39 2022] DELETE /identity/v3/users/4871205670f640499040fd5d9fdda3d9 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:40:39.838342 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9290dde-013b-4ad0-bd5b-65c873f5e472 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:39.839046 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9290dde-013b-4ad0-bd5b-65c873f5e472 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:39.839046 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9290dde-013b-4ad0-bd5b-65c873f5e472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:39.858540 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-d9290dde-013b-4ad0-bd5b-65c873f5e472 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:40:39.859784 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-d9290dde-013b-4ad0-bd5b-65c873f5e472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:39.928093 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1000/2021] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:40:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:40:39.953347 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e897d2eb-d96f-4fdc-b566-0b33c716a417 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:39.956234 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-e897d2eb-d96f-4fdc-b566-0b33c716a417 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:40.025553 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e897d2eb-d96f-4fdc-b566-0b33c716a417 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:40.029680 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e5187b-a330-42b5-bb47-c541dc7f66de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:40.030514 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e5187b-a330-42b5-bb47-c541dc7f66de service neutron] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:40.030803 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e5187b-a330-42b5-bb47-c541dc7f66de service neutron] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:40.031076 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e5187b-a330-42b5-bb47-c541dc7f66de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:40.042724 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e5187b-a330-42b5-bb47-c541dc7f66de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:40.044051 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e5187b-a330-42b5-bb47-c541dc7f66de service neutron] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:40.054323 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1022/2022] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:40:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:40:40.512652 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9290dde-013b-4ad0-bd5b-65c873f5e472 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:40.514336 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-d9290dde-013b-4ad0-bd5b-65c873f5e472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:40.528087 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9290dde-013b-4ad0-bd5b-65c873f5e472 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:40.534949 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0532145-1044-416b-a045-fff1d19db5f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:40.535831 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0532145-1044-416b-a045-fff1d19db5f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:40.536141 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0532145-1044-416b-a045-fff1d19db5f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:40.536545 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0532145-1044-416b-a045-fff1d19db5f0 admin admin] PATH_INFO: `/v3/projects/ad0b61f7690f45118ecd28f7dbc7feea` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:40.541472 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0532145-1044-416b-a045-fff1d19db5f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad0b61f7690f45118ecd28f7dbc7feea)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:40.541472 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0532145-1044-416b-a045-fff1d19db5f0 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:40.656753 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-b0532145-1044-416b-a045-fff1d19db5f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad0b61f7690f45118ecd28f7dbc7feea', 'request_id': 'req-b0532145-1044-416b-a045-fff1d19db5f0'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:40.750999 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-b0532145-1044-416b-a045-fff1d19db5f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad0b61f7690f45118ecd28f7dbc7feea', 'request_id': 'req-b0532145-1044-416b-a045-fff1d19db5f0'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:40.758794 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-b0532145-1044-416b-a045-fff1d19db5f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad0b61f7690f45118ecd28f7dbc7feea', 'request_id': 'req-b0532145-1044-416b-a045-fff1d19db5f0'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:40.770984 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-b0532145-1044-416b-a045-fff1d19db5f0 admin admin] The token cache is being invalidate because project ad0b61f7690f45118ecd28f7dbc7feea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:40:40.771277 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-b0532145-1044-416b-a045-fff1d19db5f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:40.772000 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-b0532145-1044-416b-a045-fff1d19db5f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:40.773702 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1001/2023] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:40:40 2022] DELETE /identity/v3/projects/ad0b61f7690f45118ecd28f7dbc7feea => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:40:40.783419 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e5187b-a330-42b5-bb47-c541dc7f66de service neutron] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:40.797042 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e5187b-a330-42b5-bb47-c541dc7f66de service neutron] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:40.856412 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28d1895-7376-4448-8d12-8ceb2e66723e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:40.857601 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28d1895-7376-4448-8d12-8ceb2e66723e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:40.858129 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28d1895-7376-4448-8d12-8ceb2e66723e admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:40.858338 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28d1895-7376-4448-8d12-8ceb2e66723e admin admin] PATH_INFO: `/v3/users/6636a875310e4d869860a805c0397275` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:40.859862 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28d1895-7376-4448-8d12-8ceb2e66723e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6636a875310e4d869860a805c0397275)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:40.862355 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28d1895-7376-4448-8d12-8ceb2e66723e admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:41.077298 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-e28d1895-7376-4448-8d12-8ceb2e66723e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6636a875310e4d869860a805c0397275'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:41.078199 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-e28d1895-7376-4448-8d12-8ceb2e66723e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6636a875310e4d869860a805c0397275'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:41.079103 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-e28d1895-7376-4448-8d12-8ceb2e66723e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6636a875310e4d869860a805c0397275'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:41.088301 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-e28d1895-7376-4448-8d12-8ceb2e66723e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6636a875310e4d869860a805c0397275'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:41.103363 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:40:41.106978 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-e28d1895-7376-4448-8d12-8ceb2e66723e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6636a875310e4d869860a805c0397275'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:41.274841 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1023/2024] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:40:40 2022] DELETE /identity/v3/users/6636a875310e4d869860a805c0397275 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:40:41.845118 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0532145-1044-416b-a045-fff1d19db5f0 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:41.846790 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-b0532145-1044-416b-a045-fff1d19db5f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:41.859310 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0532145-1044-416b-a045-fff1d19db5f0 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:41.951381 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0590746a-0441-40f5-85cb-2b9eaf45a824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:41.952490 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0590746a-0441-40f5-85cb-2b9eaf45a824 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:41.952785 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0590746a-0441-40f5-85cb-2b9eaf45a824 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:41.953066 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0590746a-0441-40f5-85cb-2b9eaf45a824 admin admin] PATH_INFO: `/v3/projects/c5b9e8af81dc49a19275553b57e916fd` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:41.954299 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0590746a-0441-40f5-85cb-2b9eaf45a824 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5b9e8af81dc49a19275553b57e916fd)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:41.958374 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0590746a-0441-40f5-85cb-2b9eaf45a824 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:41.978781 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:40:42.072822 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-0590746a-0441-40f5-85cb-2b9eaf45a824 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5b9e8af81dc49a19275553b57e916fd', 'request_id': 'req-0590746a-0441-40f5-85cb-2b9eaf45a824'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:42.145213 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-0590746a-0441-40f5-85cb-2b9eaf45a824 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5b9e8af81dc49a19275553b57e916fd', 'request_id': 'req-0590746a-0441-40f5-85cb-2b9eaf45a824'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:42.151741 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-0590746a-0441-40f5-85cb-2b9eaf45a824 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5b9e8af81dc49a19275553b57e916fd', 'request_id': 'req-0590746a-0441-40f5-85cb-2b9eaf45a824'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:42.162789 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-0590746a-0441-40f5-85cb-2b9eaf45a824 admin admin] The token cache is being invalidate because project c5b9e8af81dc49a19275553b57e916fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:40:42.163076 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-0590746a-0441-40f5-85cb-2b9eaf45a824 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:42.163706 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-0590746a-0441-40f5-85cb-2b9eaf45a824 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:42.165310 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1002/2025] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:40:41 2022] DELETE /identity/v3/projects/c5b9e8af81dc49a19275553b57e916fd => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:40:42.170441 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b2233d-760c-4efc-821b-9cb1bb675f2f None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:42.170896 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b2233d-760c-4efc-821b-9cb1bb675f2f None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:42.171019 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b2233d-760c-4efc-821b-9cb1bb675f2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:42.189700 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-b9b2233d-760c-4efc-821b-9cb1bb675f2f None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:40:42.201752 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-b9b2233d-760c-4efc-821b-9cb1bb675f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:42.257822 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1024/2026] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:40:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:40:42.259424 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d40a90-526a-4be4-ac0e-05c89df1c023 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:42.259724 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d40a90-526a-4be4-ac0e-05c89df1c023 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:42.260010 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d40a90-526a-4be4-ac0e-05c89df1c023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:42.289374 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-f4d40a90-526a-4be4-ac0e-05c89df1c023 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:40:42.291356 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-f4d40a90-526a-4be4-ac0e-05c89df1c023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:42.301270 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1003/2027] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:40:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:40:42.305461 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b2233d-760c-4efc-821b-9cb1bb675f2f None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:42.316261 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b2233d-760c-4efc-821b-9cb1bb675f2f None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:42.321341 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec5181b-a459-4693-8658-4a29856c65e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:42.322080 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec5181b-a459-4693-8658-4a29856c65e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:42.322370 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec5181b-a459-4693-8658-4a29856c65e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:42.325087 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec5181b-a459-4693-8658-4a29856c65e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:42.325087 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec5181b-a459-4693-8658-4a29856c65e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:40:42.325087 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec5181b-a459-4693-8658-4a29856c65e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:42.325515 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec5181b-a459-4693-8658-4a29856c65e5 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:42.340056 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1025/2028] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jun 7 14:40:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:40:42.344763 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d40a90-526a-4be4-ac0e-05c89df1c023 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:42.354369 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d40a90-526a-4be4-ac0e-05c89df1c023 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:42.359428 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db12e831-78b5-4ae9-84d8-ca9fa38be0a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:42.360078 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db12e831-78b5-4ae9-84d8-ca9fa38be0a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:42.360357 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db12e831-78b5-4ae9-84d8-ca9fa38be0a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:42.362871 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db12e831-78b5-4ae9-84d8-ca9fa38be0a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:42.362871 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db12e831-78b5-4ae9-84d8-ca9fa38be0a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:42.363268 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db12e831-78b5-4ae9-84d8-ca9fa38be0a0 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:42.459954 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1004/2029] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:40:42 2022] POST /identity/v3/projects => generated 379 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:40:42.462343 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec5181b-a459-4693-8658-4a29856c65e5 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:42.477187 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec5181b-a459-4693-8658-4a29856c65e5 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:42.535393 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c2bb27-ea95-446d-ad78-54d93e5e3c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:42.536115 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c2bb27-ea95-446d-ad78-54d93e5e3c59 admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:42.536436 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c2bb27-ea95-446d-ad78-54d93e5e3c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:42.536770 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c2bb27-ea95-446d-ad78-54d93e5e3c59 admin admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:42.537473 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c2bb27-ea95-446d-ad78-54d93e5e3c59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:42.539403 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c2bb27-ea95-446d-ad78-54d93e5e3c59 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:42.621542 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1026/2030] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:40:42 2022] POST /identity/v3/users => generated 290 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:40:42.627966 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db12e831-78b5-4ae9-84d8-ca9fa38be0a0 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:42.638661 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db12e831-78b5-4ae9-84d8-ca9fa38be0a0 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:42.643377 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576c2400-6a50-4fc3-87cd-576eba9cd79b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:42.644151 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c2400-6a50-4fc3-87cd-576eba9cd79b admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:42.644405 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c2400-6a50-4fc3-87cd-576eba9cd79b admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:42.644688 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c2400-6a50-4fc3-87cd-576eba9cd79b admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:42.645413 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576c2400-6a50-4fc3-87cd-576eba9cd79b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:42.652345 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576c2400-6a50-4fc3-87cd-576eba9cd79b admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:42.658062 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1005/2031] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:40:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:40:42.662439 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c2bb27-ea95-446d-ad78-54d93e5e3c59 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:42.674593 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c2bb27-ea95-446d-ad78-54d93e5e3c59 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:42.681790 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480013b5-da81-4b1c-8c54-367e6e3d7798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:42.682465 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480013b5-da81-4b1c-8c54-367e6e3d7798 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:42.682802 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480013b5-da81-4b1c-8c54-367e6e3d7798 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:42.683088 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480013b5-da81-4b1c-8c54-367e6e3d7798 admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:42.683835 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480013b5-da81-4b1c-8c54-367e6e3d7798 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:42.685478 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480013b5-da81-4b1c-8c54-367e6e3d7798 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:42.696939 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1027/2032] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:40:42 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 14:40:42.701183 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576c2400-6a50-4fc3-87cd-576eba9cd79b admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:42.712829 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576c2400-6a50-4fc3-87cd-576eba9cd79b admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:42.718049 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004cb795-9292-4601-a44e-d79c08b77152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:42.718941 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004cb795-9292-4601-a44e-d79c08b77152 admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:42.719228 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004cb795-9292-4601-a44e-d79c08b77152 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:42.719509 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004cb795-9292-4601-a44e-d79c08b77152 admin admin] PATH_INFO: `/v3/projects/0720336fee014b5694248abaa1f45153/users/9447bd82da3d47b9a6980f4bddf88ab3/roles/3771341b3be641139e752aab690397a4` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:42.734628 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004cb795-9292-4601-a44e-d79c08b77152 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0720336fee014b5694248abaa1f45153, user_id=9447bd82da3d47b9a6980f4bddf88ab3, role_id=3771341b3be641139e752aab690397a4)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:42.737924 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004cb795-9292-4601-a44e-d79c08b77152 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:42.791331 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1006/2033] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:40:42 2022] PUT /identity/v3/projects/0720336fee014b5694248abaa1f45153/users/9447bd82da3d47b9a6980f4bddf88ab3/roles/3771341b3be641139e752aab690397a4 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:40:42.802248 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a3f74-1e00-4b67-9dd5-5ee1f1496a44 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:42.802541 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a3f74-1e00-4b67-9dd5-5ee1f1496a44 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:42.802822 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a3f74-1e00-4b67-9dd5-5ee1f1496a44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:42.832517 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-188a3f74-1e00-4b67-9dd5-5ee1f1496a44 None None] MFA Rules not processed for user `9447bd82da3d47b9a6980f4bddf88ab3`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:40:42.833764 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-188a3f74-1e00-4b67-9dd5-5ee1f1496a44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:42.881147 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:40:42.904511 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1028/2034] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:40:42 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:40:42.909020 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004cb795-9292-4601-a44e-d79c08b77152 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:42.995930 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004cb795-9292-4601-a44e-d79c08b77152 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:43.003766 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91400a8f-d249-4a6a-b745-a4625f0c90ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:43.004649 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91400a8f-d249-4a6a-b745-a4625f0c90ea admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:43.005091 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91400a8f-d249-4a6a-b745-a4625f0c90ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:43.005387 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91400a8f-d249-4a6a-b745-a4625f0c90ea admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:43.006086 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91400a8f-d249-4a6a-b745-a4625f0c90ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:43.008132 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91400a8f-d249-4a6a-b745-a4625f0c90ea admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:43.093003 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1007/2035] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:40:42 2022] POST /identity/v3/projects => generated 379 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:40:43.094150 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188a3f74-1e00-4b67-9dd5-5ee1f1496a44 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:43.105408 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188a3f74-1e00-4b67-9dd5-5ee1f1496a44 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:43.167024 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5cb652-2816-41bb-8524-c791f58d4112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:43.167703 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5cb652-2816-41bb-8524-c791f58d4112 admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:43.167989 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5cb652-2816-41bb-8524-c791f58d4112 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:43.168279 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5cb652-2816-41bb-8524-c791f58d4112 admin admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:43.168991 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5cb652-2816-41bb-8524-c791f58d4112 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:43.170932 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5cb652-2816-41bb-8524-c791f58d4112 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:43.246507 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1029/2036] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:40:43 2022] POST /identity/v3/users => generated 296 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:40:43.248150 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91400a8f-d249-4a6a-b745-a4625f0c90ea admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:43.260448 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91400a8f-d249-4a6a-b745-a4625f0c90ea admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:43.265420 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230e23f4-e591-4a63-97e7-1a8735c584de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:43.266090 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230e23f4-e591-4a63-97e7-1a8735c584de admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:43.266378 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230e23f4-e591-4a63-97e7-1a8735c584de admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:43.266656 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230e23f4-e591-4a63-97e7-1a8735c584de admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:43.267383 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230e23f4-e591-4a63-97e7-1a8735c584de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:43.269102 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230e23f4-e591-4a63-97e7-1a8735c584de admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:43.282322 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1008/2037] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:40:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:40:43.286610 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5cb652-2816-41bb-8524-c791f58d4112 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:43.297127 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5cb652-2816-41bb-8524-c791f58d4112 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:43.302738 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81b3a8e-3b44-4b94-8c62-9abd86581f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:43.307186 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81b3a8e-3b44-4b94-8c62-9abd86581f2a admin admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:43.307186 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81b3a8e-3b44-4b94-8c62-9abd86581f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:43.307186 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81b3a8e-3b44-4b94-8c62-9abd86581f2a admin admin] PATH_INFO: `/v3/projects/5a10da05496c4eb4bcf7ca12c77cd96b/users/d73f0b0ddfb84a58b8e785cbf61f636a/roles/9baff2b18f2649f7b757ff2e1f63c853` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:43.321769 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81b3a8e-3b44-4b94-8c62-9abd86581f2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a10da05496c4eb4bcf7ca12c77cd96b, user_id=d73f0b0ddfb84a58b8e785cbf61f636a, role_id=9baff2b18f2649f7b757ff2e1f63c853)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:43.324484 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81b3a8e-3b44-4b94-8c62-9abd86581f2a admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:43.393760 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1030/2038] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:40:43 2022] PUT /identity/v3/projects/5a10da05496c4eb4bcf7ca12c77cd96b/users/d73f0b0ddfb84a58b8e785cbf61f636a/roles/9baff2b18f2649f7b757ff2e1f63c853 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:40:43.393916 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230e23f4-e591-4a63-97e7-1a8735c584de admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:43.403751 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230e23f4-e591-4a63-97e7-1a8735c584de admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:43.417284 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:40:43.472636 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da589bf0-6073-4dbb-b3dc-5fd59d5b9c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:43.473664 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da589bf0-6073-4dbb-b3dc-5fd59d5b9c33 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:43.473969 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da589bf0-6073-4dbb-b3dc-5fd59d5b9c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:43.474255 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da589bf0-6073-4dbb-b3dc-5fd59d5b9c33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:43.474998 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da589bf0-6073-4dbb-b3dc-5fd59d5b9c33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:43.476775 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da589bf0-6073-4dbb-b3dc-5fd59d5b9c33 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:43.494421 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1009/2039] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:40:43 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:40:43.496002 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81b3a8e-3b44-4b94-8c62-9abd86581f2a admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:43.508517 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81b3a8e-3b44-4b94-8c62-9abd86581f2a admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:43.514963 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd10338c-2fd9-48da-85b9-f271cf606247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:43.514963 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd10338c-2fd9-48da-85b9-f271cf606247 admin admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:43.514963 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd10338c-2fd9-48da-85b9-f271cf606247 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:43.515286 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd10338c-2fd9-48da-85b9-f271cf606247 admin admin] PATH_INFO: `/v3/projects/5a10da05496c4eb4bcf7ca12c77cd96b/users/d73f0b0ddfb84a58b8e785cbf61f636a/roles/9baff2b18f2649f7b757ff2e1f63c853` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:43.517507 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd10338c-2fd9-48da-85b9-f271cf606247 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a10da05496c4eb4bcf7ca12c77cd96b, user_id=d73f0b0ddfb84a58b8e785cbf61f636a, role_id=9baff2b18f2649f7b757ff2e1f63c853)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:43.520140 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd10338c-2fd9-48da-85b9-f271cf606247 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:43.532061 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1031/2040] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:40:43 2022] PUT /identity/v3/projects/5a10da05496c4eb4bcf7ca12c77cd96b/users/d73f0b0ddfb84a58b8e785cbf61f636a/roles/9baff2b18f2649f7b757ff2e1f63c853 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:40:43.543646 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7784e37c-c00b-4bbc-8785-bc6d8f2be9be None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:43.543939 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7784e37c-c00b-4bbc-8785-bc6d8f2be9be None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:43.544218 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7784e37c-c00b-4bbc-8785-bc6d8f2be9be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:43.578440 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-7784e37c-c00b-4bbc-8785-bc6d8f2be9be None None] MFA Rules not processed for user `d73f0b0ddfb84a58b8e785cbf61f636a`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:40:43.578440 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-7784e37c-c00b-4bbc-8785-bc6d8f2be9be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:43.651531 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1010/2041] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:40:43 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:40:43.662904 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd10338c-2fd9-48da-85b9-f271cf606247 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:43.665218 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-cd10338c-2fd9-48da-85b9-f271cf606247 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:43.723383 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd10338c-2fd9-48da-85b9-f271cf606247 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:43.730261 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10436984-03e6-42e7-9a36-c2fdf37425e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d81f597cb44c56a966537089bcce4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:43.730973 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10436984-03e6-42e7-9a36-c2fdf37425e6 service cinder] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:43.731282 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10436984-03e6-42e7-9a36-c2fdf37425e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:43.731558 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10436984-03e6-42e7-9a36-c2fdf37425e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:43.742321 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10436984-03e6-42e7-9a36-c2fdf37425e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:43.743745 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10436984-03e6-42e7-9a36-c2fdf37425e6 service cinder] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:43.753219 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1032/2042] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:40:43 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:40:44.309201 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7784e37c-c00b-4bbc-8785-bc6d8f2be9be None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:44.374089 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7784e37c-c00b-4bbc-8785-bc6d8f2be9be None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:44.379256 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54146f32-e455-4400-9113-5cde55978506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:44.380134 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54146f32-e455-4400-9113-5cde55978506 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:44.380449 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54146f32-e455-4400-9113-5cde55978506 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:44.380735 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54146f32-e455-4400-9113-5cde55978506 admin admin] PATH_INFO: `/v3/users/9447bd82da3d47b9a6980f4bddf88ab3` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:44.381940 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54146f32-e455-4400-9113-5cde55978506 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9447bd82da3d47b9a6980f4bddf88ab3)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:44.383988 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54146f32-e455-4400-9113-5cde55978506 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:44.568391 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:40:44.577517 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-54146f32-e455-4400-9113-5cde55978506 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9447bd82da3d47b9a6980f4bddf88ab3'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:44.578181 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-54146f32-e455-4400-9113-5cde55978506 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9447bd82da3d47b9a6980f4bddf88ab3'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:44.578805 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-54146f32-e455-4400-9113-5cde55978506 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9447bd82da3d47b9a6980f4bddf88ab3'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:44.586146 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-54146f32-e455-4400-9113-5cde55978506 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9447bd82da3d47b9a6980f4bddf88ab3'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:44.605135 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-54146f32-e455-4400-9113-5cde55978506 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9447bd82da3d47b9a6980f4bddf88ab3'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:44.653666 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1011/2043] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:40:44 2022] DELETE /identity/v3/users/9447bd82da3d47b9a6980f4bddf88ab3 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:40:44.658320 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5953452c-05e0-4ac6-b03a-883b01ab250f None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:44.658828 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5953452c-05e0-4ac6-b03a-883b01ab250f None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:44.658922 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5953452c-05e0-4ac6-b03a-883b01ab250f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:44.677020 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-5953452c-05e0-4ac6-b03a-883b01ab250f None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:40:44.678694 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-5953452c-05e0-4ac6-b03a-883b01ab250f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:44.718396 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:40:44.756797 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1033/2044] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:40:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:40:44.773545 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54146f32-e455-4400-9113-5cde55978506 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:44.773689 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5953452c-05e0-4ac6-b03a-883b01ab250f None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:44.774987 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-5953452c-05e0-4ac6-b03a-883b01ab250f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:44.775098 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-54146f32-e455-4400-9113-5cde55978506 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:44.788959 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5953452c-05e0-4ac6-b03a-883b01ab250f None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:44.794540 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b45772-9f43-4fc0-8982-6d03aad0b46d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:44.796035 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b45772-9f43-4fc0-8982-6d03aad0b46d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:44.796035 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b45772-9f43-4fc0-8982-6d03aad0b46d admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:44.796252 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b45772-9f43-4fc0-8982-6d03aad0b46d admin admin] PATH_INFO: `/v3/users/ca63a39dcc00445b984d9455294f8ff1` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:44.797432 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b45772-9f43-4fc0-8982-6d03aad0b46d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca63a39dcc00445b984d9455294f8ff1)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:44.800055 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b45772-9f43-4fc0-8982-6d03aad0b46d admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:44.842531 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54146f32-e455-4400-9113-5cde55978506 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:44.846209 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf052b06-0ca4-4b3b-ad51-9626bf69df1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:44.846860 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf052b06-0ca4-4b3b-ad51-9626bf69df1f service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:44.847143 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf052b06-0ca4-4b3b-ad51-9626bf69df1f service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:44.847420 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf052b06-0ca4-4b3b-ad51-9626bf69df1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:44.856832 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf052b06-0ca4-4b3b-ad51-9626bf69df1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:44.861536 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf052b06-0ca4-4b3b-ad51-9626bf69df1f service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:44.870144 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1012/2045] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:40:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:40:45.012116 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-28b45772-9f43-4fc0-8982-6d03aad0b46d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca63a39dcc00445b984d9455294f8ff1'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:45.012762 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-28b45772-9f43-4fc0-8982-6d03aad0b46d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca63a39dcc00445b984d9455294f8ff1'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:45.013646 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-28b45772-9f43-4fc0-8982-6d03aad0b46d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca63a39dcc00445b984d9455294f8ff1'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:45.023377 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-28b45772-9f43-4fc0-8982-6d03aad0b46d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca63a39dcc00445b984d9455294f8ff1'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:45.044585 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-28b45772-9f43-4fc0-8982-6d03aad0b46d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca63a39dcc00445b984d9455294f8ff1'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:45.148428 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1034/2046] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:40:44 2022] DELETE /identity/v3/users/ca63a39dcc00445b984d9455294f8ff1 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:40:45.158895 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42858e3d-49f1-4ed8-b0a9-12f6d76851f0 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:45.159430 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42858e3d-49f1-4ed8-b0a9-12f6d76851f0 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:45.159590 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42858e3d-49f1-4ed8-b0a9-12f6d76851f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:45.182864 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-42858e3d-49f1-4ed8-b0a9-12f6d76851f0 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:40:45.184306 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-42858e3d-49f1-4ed8-b0a9-12f6d76851f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:45.255413 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1013/2047] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:40:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:40:45.266407 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b45772-9f43-4fc0-8982-6d03aad0b46d admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:45.268506 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-28b45772-9f43-4fc0-8982-6d03aad0b46d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:45.341541 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b45772-9f43-4fc0-8982-6d03aad0b46d admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:45.345999 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2902dae7-0c9e-4268-b886-536f5057b322 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:45.346670 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2902dae7-0c9e-4268-b886-536f5057b322 service neutron] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:45.346960 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2902dae7-0c9e-4268-b886-536f5057b322 service neutron] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:45.347242 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2902dae7-0c9e-4268-b886-536f5057b322 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:45.359288 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42858e3d-49f1-4ed8-b0a9-12f6d76851f0 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:45.360870 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-42858e3d-49f1-4ed8-b0a9-12f6d76851f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:45.368148 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2902dae7-0c9e-4268-b886-536f5057b322 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:45.368784 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2902dae7-0c9e-4268-b886-536f5057b322 service neutron] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:45.385021 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42858e3d-49f1-4ed8-b0a9-12f6d76851f0 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:45.391196 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cafa496-5c8b-426d-b983-a0851021b2cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:45.393683 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cafa496-5c8b-426d-b983-a0851021b2cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:45.394224 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cafa496-5c8b-426d-b983-a0851021b2cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:45.395058 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cafa496-5c8b-426d-b983-a0851021b2cb admin admin] PATH_INFO: `/v3/projects/0720336fee014b5694248abaa1f45153` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:45.397212 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cafa496-5c8b-426d-b983-a0851021b2cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=0720336fee014b5694248abaa1f45153)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:45.399511 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cafa496-5c8b-426d-b983-a0851021b2cb admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:45.404345 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1035/2048] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:40:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:40:45.541923 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-9cafa496-5c8b-426d-b983-a0851021b2cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0720336fee014b5694248abaa1f45153', 'request_id': 'req-9cafa496-5c8b-426d-b983-a0851021b2cb'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:45.546798 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:40:45.614537 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-9cafa496-5c8b-426d-b983-a0851021b2cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0720336fee014b5694248abaa1f45153', 'request_id': 'req-9cafa496-5c8b-426d-b983-a0851021b2cb'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:45.623986 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-9cafa496-5c8b-426d-b983-a0851021b2cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0720336fee014b5694248abaa1f45153', 'request_id': 'req-9cafa496-5c8b-426d-b983-a0851021b2cb'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:45.636937 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-9cafa496-5c8b-426d-b983-a0851021b2cb admin admin] The token cache is being invalidate because project 0720336fee014b5694248abaa1f45153 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:40:45.637208 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-9cafa496-5c8b-426d-b983-a0851021b2cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:45.637916 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-9cafa496-5c8b-426d-b983-a0851021b2cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:45.639823 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1014/2049] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:40:45 2022] DELETE /identity/v3/projects/0720336fee014b5694248abaa1f45153 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:40:45.645717 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2902dae7-0c9e-4268-b886-536f5057b322 service neutron] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:45.661399 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2902dae7-0c9e-4268-b886-536f5057b322 service neutron] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:45.732586 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:40:45.753015 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa30d063-eb9d-4300-9e9d-73d8f40b7737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:45.753928 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa30d063-eb9d-4300-9e9d-73d8f40b7737 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:45.754205 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa30d063-eb9d-4300-9e9d-73d8f40b7737 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:45.754935 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa30d063-eb9d-4300-9e9d-73d8f40b7737 admin admin] PATH_INFO: `/v3/users/d73f0b0ddfb84a58b8e785cbf61f636a` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:45.756700 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa30d063-eb9d-4300-9e9d-73d8f40b7737 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d73f0b0ddfb84a58b8e785cbf61f636a)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:45.760125 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa30d063-eb9d-4300-9e9d-73d8f40b7737 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:45.867022 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cafa496-5c8b-426d-b983-a0851021b2cb admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:45.868590 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-9cafa496-5c8b-426d-b983-a0851021b2cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:45.882185 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cafa496-5c8b-426d-b983-a0851021b2cb admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:45.887011 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44099802-a669-44f4-bdc1-c4c99e22446f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:45.887889 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44099802-a669-44f4-bdc1-c4c99e22446f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:45.888176 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44099802-a669-44f4-bdc1-c4c99e22446f admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:45.888748 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44099802-a669-44f4-bdc1-c4c99e22446f admin admin] PATH_INFO: `/v3/projects/7123e05a6a4143e6b4a24f2d9322508c` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:45.889962 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44099802-a669-44f4-bdc1-c4c99e22446f admin admin] RBAC: Authorizing `identity:delete_project(project_id=7123e05a6a4143e6b4a24f2d9322508c)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:45.891877 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44099802-a669-44f4-bdc1-c4c99e22446f admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:45.977664 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-fa30d063-eb9d-4300-9e9d-73d8f40b7737 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd73f0b0ddfb84a58b8e785cbf61f636a'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:45.978618 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-fa30d063-eb9d-4300-9e9d-73d8f40b7737 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd73f0b0ddfb84a58b8e785cbf61f636a'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:45.979410 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-fa30d063-eb9d-4300-9e9d-73d8f40b7737 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd73f0b0ddfb84a58b8e785cbf61f636a'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:45.986293 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-44099802-a669-44f4-bdc1-c4c99e22446f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7123e05a6a4143e6b4a24f2d9322508c', 'request_id': 'req-44099802-a669-44f4-bdc1-c4c99e22446f'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:45.986943 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-fa30d063-eb9d-4300-9e9d-73d8f40b7737 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd73f0b0ddfb84a58b8e785cbf61f636a'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:45.998884 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-fa30d063-eb9d-4300-9e9d-73d8f40b7737 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd73f0b0ddfb84a58b8e785cbf61f636a'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:46.065049 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-44099802-a669-44f4-bdc1-c4c99e22446f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7123e05a6a4143e6b4a24f2d9322508c', 'request_id': 'req-44099802-a669-44f4-bdc1-c4c99e22446f'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:46.070919 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-44099802-a669-44f4-bdc1-c4c99e22446f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7123e05a6a4143e6b4a24f2d9322508c', 'request_id': 'req-44099802-a669-44f4-bdc1-c4c99e22446f'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:46.089200 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-44099802-a669-44f4-bdc1-c4c99e22446f admin admin] The token cache is being invalidate because project 7123e05a6a4143e6b4a24f2d9322508c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:40:46.089495 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-44099802-a669-44f4-bdc1-c4c99e22446f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:46.090321 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-44099802-a669-44f4-bdc1-c4c99e22446f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:46.100306 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1015/2050] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:40:45 2022] DELETE /identity/v3/projects/7123e05a6a4143e6b4a24f2d9322508c => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:40:46.100306 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44099802-a669-44f4-bdc1-c4c99e22446f admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:46.100306 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-44099802-a669-44f4-bdc1-c4c99e22446f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:46.120258 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1036/2051] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:40:45 2022] DELETE /identity/v3/users/d73f0b0ddfb84a58b8e785cbf61f636a => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:40:46.188760 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44099802-a669-44f4-bdc1-c4c99e22446f admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:46.257746 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a4ba12-38d3-4c7d-b8ff-5fe400a2c022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:46.258640 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a4ba12-38d3-4c7d-b8ff-5fe400a2c022 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:46.259102 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a4ba12-38d3-4c7d-b8ff-5fe400a2c022 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:46.259394 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a4ba12-38d3-4c7d-b8ff-5fe400a2c022 admin admin] PATH_INFO: `/v3/users/a04507052803403ba5a5f29b416220fa` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:46.260976 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a4ba12-38d3-4c7d-b8ff-5fe400a2c022 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a04507052803403ba5a5f29b416220fa)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:46.266378 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a4ba12-38d3-4c7d-b8ff-5fe400a2c022 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:46.281351 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:40:46.439367 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-90a4ba12-38d3-4c7d-b8ff-5fe400a2c022 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a04507052803403ba5a5f29b416220fa'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:46.440058 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-90a4ba12-38d3-4c7d-b8ff-5fe400a2c022 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a04507052803403ba5a5f29b416220fa'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:46.449451 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-90a4ba12-38d3-4c7d-b8ff-5fe400a2c022 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a04507052803403ba5a5f29b416220fa'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:46.456842 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-90a4ba12-38d3-4c7d-b8ff-5fe400a2c022 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a04507052803403ba5a5f29b416220fa'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:46.473389 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-90a4ba12-38d3-4c7d-b8ff-5fe400a2c022 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a04507052803403ba5a5f29b416220fa'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:46.535864 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1016/2052] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:40:46 2022] DELETE /identity/v3/users/a04507052803403ba5a5f29b416220fa => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:40:46.555117 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa30d063-eb9d-4300-9e9d-73d8f40b7737 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:46.556916 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-fa30d063-eb9d-4300-9e9d-73d8f40b7737 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:46.596115 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa30d063-eb9d-4300-9e9d-73d8f40b7737 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:46.658613 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556b8b82-e1ea-47b5-84d7-34d6018018a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:46.659282 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556b8b82-e1ea-47b5-84d7-34d6018018a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:46.659570 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556b8b82-e1ea-47b5-84d7-34d6018018a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:46.659839 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556b8b82-e1ea-47b5-84d7-34d6018018a6 admin admin] PATH_INFO: `/v3/projects/5a10da05496c4eb4bcf7ca12c77cd96b` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:46.661182 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556b8b82-e1ea-47b5-84d7-34d6018018a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a10da05496c4eb4bcf7ca12c77cd96b)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:46.663491 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556b8b82-e1ea-47b5-84d7-34d6018018a6 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:46.764940 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-556b8b82-e1ea-47b5-84d7-34d6018018a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a10da05496c4eb4bcf7ca12c77cd96b', 'request_id': 'req-556b8b82-e1ea-47b5-84d7-34d6018018a6'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:46.825490 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-556b8b82-e1ea-47b5-84d7-34d6018018a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a10da05496c4eb4bcf7ca12c77cd96b', 'request_id': 'req-556b8b82-e1ea-47b5-84d7-34d6018018a6'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:46.834268 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-556b8b82-e1ea-47b5-84d7-34d6018018a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a10da05496c4eb4bcf7ca12c77cd96b', 'request_id': 'req-556b8b82-e1ea-47b5-84d7-34d6018018a6'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:46.847942 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-556b8b82-e1ea-47b5-84d7-34d6018018a6 admin admin] The token cache is being invalidate because project 5a10da05496c4eb4bcf7ca12c77cd96b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:40:46.848361 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-556b8b82-e1ea-47b5-84d7-34d6018018a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:46.849009 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-556b8b82-e1ea-47b5-84d7-34d6018018a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:46.850740 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1037/2053] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:40:46 2022] DELETE /identity/v3/projects/5a10da05496c4eb4bcf7ca12c77cd96b => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:40:46.857056 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c828c5-2d2b-4fa4-8bec-e0a4f63b1354 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:46.866395 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c828c5-2d2b-4fa4-8bec-e0a4f63b1354 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:46.866395 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c828c5-2d2b-4fa4-8bec-e0a4f63b1354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:46.894151 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-a2c828c5-2d2b-4fa4-8bec-e0a4f63b1354 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:40:46.895379 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-a2c828c5-2d2b-4fa4-8bec-e0a4f63b1354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:46.997941 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1017/2054] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:40:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:40:47.004628 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db6eb0c-4938-4e3a-8dcf-d762bb893dff None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:47.005130 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db6eb0c-4938-4e3a-8dcf-d762bb893dff None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:47.005632 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db6eb0c-4938-4e3a-8dcf-d762bb893dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:47.024949 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:40:47.039495 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-7db6eb0c-4938-4e3a-8dcf-d762bb893dff None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:40:47.047148 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-7db6eb0c-4938-4e3a-8dcf-d762bb893dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:47.056607 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1038/2055] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:40:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:40:47.057583 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db6eb0c-4938-4e3a-8dcf-d762bb893dff None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:47.061720 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c828c5-2d2b-4fa4-8bec-e0a4f63b1354 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:47.061720 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-a2c828c5-2d2b-4fa4-8bec-e0a4f63b1354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:47.069819 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db6eb0c-4938-4e3a-8dcf-d762bb893dff None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:47.075034 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c828c5-2d2b-4fa4-8bec-e0a4f63b1354 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:47.085226 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed374395-e4f4-41bb-95e4-b5fce6496f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:47.085976 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3c245a-2d9f-486b-9944-4abbaa3b9ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:47.086692 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed374395-e4f4-41bb-95e4-b5fce6496f80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:47.086993 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed374395-e4f4-41bb-95e4-b5fce6496f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:47.087277 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed374395-e4f4-41bb-95e4-b5fce6496f80 admin admin] PATH_INFO: `/v3/users/d800b8efe792477bac3ca063920a4a9a` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:47.087605 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3c245a-2d9f-486b-9944-4abbaa3b9ce2 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:47.088233 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3c245a-2d9f-486b-9944-4abbaa3b9ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:47.088781 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3c245a-2d9f-486b-9944-4abbaa3b9ce2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:47.089134 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed374395-e4f4-41bb-95e4-b5fce6496f80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d800b8efe792477bac3ca063920a4a9a)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:47.090295 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3c245a-2d9f-486b-9944-4abbaa3b9ce2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:40:47.092420 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3c245a-2d9f-486b-9944-4abbaa3b9ce2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:47.092571 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed374395-e4f4-41bb-95e4-b5fce6496f80 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:47.092667 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3c245a-2d9f-486b-9944-4abbaa3b9ce2 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:47.106094 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1039/2056] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jun 7 14:40:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:40:47.113953 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3c245a-2d9f-486b-9944-4abbaa3b9ce2 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:47.128332 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3c245a-2d9f-486b-9944-4abbaa3b9ce2 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:47.135990 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500dfc4d-e2c6-4022-a1f9-4721d8e01283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:47.137098 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500dfc4d-e2c6-4022-a1f9-4721d8e01283 admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:47.137601 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500dfc4d-e2c6-4022-a1f9-4721d8e01283 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:47.138213 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500dfc4d-e2c6-4022-a1f9-4721d8e01283 admin admin] PATH_INFO: `/v3/projects` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:47.139477 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500dfc4d-e2c6-4022-a1f9-4721d8e01283 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:47.141472 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500dfc4d-e2c6-4022-a1f9-4721d8e01283 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:47.255587 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1040/2057] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:40:47 2022] POST /identity/v3/projects => generated 377 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:40:47.256618 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500dfc4d-e2c6-4022-a1f9-4721d8e01283 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:47.259139 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-500dfc4d-e2c6-4022-a1f9-4721d8e01283 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:47.298799 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-ed374395-e4f4-41bb-95e4-b5fce6496f80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd800b8efe792477bac3ca063920a4a9a'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:47.299919 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-ed374395-e4f4-41bb-95e4-b5fce6496f80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd800b8efe792477bac3ca063920a4a9a'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:47.300296 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-ed374395-e4f4-41bb-95e4-b5fce6496f80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd800b8efe792477bac3ca063920a4a9a'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:47.309970 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-ed374395-e4f4-41bb-95e4-b5fce6496f80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd800b8efe792477bac3ca063920a4a9a'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:47.327633 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-ed374395-e4f4-41bb-95e4-b5fce6496f80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd800b8efe792477bac3ca063920a4a9a'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:47.357846 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500dfc4d-e2c6-4022-a1f9-4721d8e01283 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:47.365545 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9a2a11-9e01-4e3e-b745-34839eea2468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:47.366451 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9a2a11-9e01-4e3e-b745-34839eea2468 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:47.366890 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9a2a11-9e01-4e3e-b745-34839eea2468 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:47.367293 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9a2a11-9e01-4e3e-b745-34839eea2468 admin admin] PATH_INFO: `/v3/projects/4103c823578e4fc0a5a96b7a4e631855` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:47.369186 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9a2a11-9e01-4e3e-b745-34839eea2468 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4103c823578e4fc0a5a96b7a4e631855)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:47.370548 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9a2a11-9e01-4e3e-b745-34839eea2468 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:47.398747 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1018/2058] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:40:47 2022] DELETE /identity/v3/users/d800b8efe792477bac3ca063920a4a9a => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:40:47.399578 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed374395-e4f4-41bb-95e4-b5fce6496f80 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:47.401275 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-ed374395-e4f4-41bb-95e4-b5fce6496f80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:47.407313 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:40:47.505717 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-db9a2a11-9e01-4e3e-b745-34839eea2468 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4103c823578e4fc0a5a96b7a4e631855', 'request_id': 'req-db9a2a11-9e01-4e3e-b745-34839eea2468'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:47.510653 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed374395-e4f4-41bb-95e4-b5fce6496f80 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:47.608592 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-db9a2a11-9e01-4e3e-b745-34839eea2468 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4103c823578e4fc0a5a96b7a4e631855', 'request_id': 'req-db9a2a11-9e01-4e3e-b745-34839eea2468'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:47.618891 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e772c8e6-118a-4370-a63a-4590889526a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:47.619594 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e772c8e6-118a-4370-a63a-4590889526a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:47.619881 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e772c8e6-118a-4370-a63a-4590889526a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:47.620158 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e772c8e6-118a-4370-a63a-4590889526a5 admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:47.620955 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e772c8e6-118a-4370-a63a-4590889526a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:47.623057 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e772c8e6-118a-4370-a63a-4590889526a5 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:47.624036 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-db9a2a11-9e01-4e3e-b745-34839eea2468 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4103c823578e4fc0a5a96b7a4e631855', 'request_id': 'req-db9a2a11-9e01-4e3e-b745-34839eea2468'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:47.644456 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-db9a2a11-9e01-4e3e-b745-34839eea2468 admin admin] The token cache is being invalidate because project 4103c823578e4fc0a5a96b7a4e631855 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:40:47.644456 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-db9a2a11-9e01-4e3e-b745-34839eea2468 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:47.644456 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-db9a2a11-9e01-4e3e-b745-34839eea2468 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:47.644957 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1041/2059] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:40:47 2022] DELETE /identity/v3/projects/4103c823578e4fc0a5a96b7a4e631855 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:40:47.649790 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d775637f-0874-42fc-b645-4f692a514ffe None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:47.650081 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d775637f-0874-42fc-b645-4f692a514ffe None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:47.650359 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d775637f-0874-42fc-b645-4f692a514ffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:47.668803 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-d775637f-0874-42fc-b645-4f692a514ffe None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:40:47.670103 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-d775637f-0874-42fc-b645-4f692a514ffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:47.689556 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1042/2060] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:40:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:40:47.691799 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f021a507-f031-4b1e-98d8-47d2ff78c0d5 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:47.692100 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f021a507-f031-4b1e-98d8-47d2ff78c0d5 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:47.692396 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f021a507-f031-4b1e-98d8-47d2ff78c0d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:47.712627 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-f021a507-f031-4b1e-98d8-47d2ff78c0d5 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:40:47.713857 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-f021a507-f031-4b1e-98d8-47d2ff78c0d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:47.720473 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1019/2061] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:40:47 2022] POST /identity/v3/users => generated 289 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:40:47.721334 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e772c8e6-118a-4370-a63a-4590889526a5 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:47.723829 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-e772c8e6-118a-4370-a63a-4590889526a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:47.727942 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1043/2062] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:40:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:40:47.730717 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f021a507-f031-4b1e-98d8-47d2ff78c0d5 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:47.730717 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-f021a507-f031-4b1e-98d8-47d2ff78c0d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:47.740366 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e772c8e6-118a-4370-a63a-4590889526a5 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:47.745761 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f021a507-f031-4b1e-98d8-47d2ff78c0d5 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:47.748316 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d424f0cb-2a67-4c71-a986-f926c2787e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:47.749457 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d424f0cb-2a67-4c71-a986-f926c2787e50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:47.749847 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d424f0cb-2a67-4c71-a986-f926c2787e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:47.750342 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d424f0cb-2a67-4c71-a986-f926c2787e50 admin admin] PATH_INFO: `/v3/projects/1bd7cee3f5704d01ae9df6d300b30004` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:47.751284 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976ab487-68e0-4f81-99d0-953fb6894758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:47.752630 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976ab487-68e0-4f81-99d0-953fb6894758 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:47.752880 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d424f0cb-2a67-4c71-a986-f926c2787e50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bd7cee3f5704d01ae9df6d300b30004)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:47.752994 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976ab487-68e0-4f81-99d0-953fb6894758 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:47.753192 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976ab487-68e0-4f81-99d0-953fb6894758 admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:47.753934 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976ab487-68e0-4f81-99d0-953fb6894758 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:47.755527 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976ab487-68e0-4f81-99d0-953fb6894758 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:47.756449 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d424f0cb-2a67-4c71-a986-f926c2787e50 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:47.772506 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1044/2063] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:40:47 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:40:47.772506 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976ab487-68e0-4f81-99d0-953fb6894758 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:47.782361 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976ab487-68e0-4f81-99d0-953fb6894758 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:47.788829 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79347ae-00d5-45a0-9dab-93c64f7a9a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:47.790008 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79347ae-00d5-45a0-9dab-93c64f7a9a2d admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:47.790886 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79347ae-00d5-45a0-9dab-93c64f7a9a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:47.790886 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79347ae-00d5-45a0-9dab-93c64f7a9a2d admin admin] PATH_INFO: `/v3/domains` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:47.793893 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79347ae-00d5-45a0-9dab-93c64f7a9a2d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:40:47.793893 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79347ae-00d5-45a0-9dab-93c64f7a9a2d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:47.794182 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79347ae-00d5-45a0-9dab-93c64f7a9a2d admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:47.812215 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1045/2064] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jun 7 14:40:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:40:47.816729 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79347ae-00d5-45a0-9dab-93c64f7a9a2d admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:47.826843 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79347ae-00d5-45a0-9dab-93c64f7a9a2d admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:47.832193 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51fe4fd-bd19-4a7c-b1d9-77a875637c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:47.832894 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51fe4fd-bd19-4a7c-b1d9-77a875637c53 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:47.833195 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51fe4fd-bd19-4a7c-b1d9-77a875637c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:47.833488 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51fe4fd-bd19-4a7c-b1d9-77a875637c53 admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:47.838678 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51fe4fd-bd19-4a7c-b1d9-77a875637c53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:47.838678 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51fe4fd-bd19-4a7c-b1d9-77a875637c53 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:47.846475 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1046/2065] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:40:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:40:47.847373 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51fe4fd-bd19-4a7c-b1d9-77a875637c53 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:47.852464 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:40:47.857981 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51fe4fd-bd19-4a7c-b1d9-77a875637c53 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:47.863543 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6d103d-443c-4505-856c-e42a667c898c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:47.863992 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6d103d-443c-4505-856c-e42a667c898c admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:47.864662 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6d103d-443c-4505-856c-e42a667c898c admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:47.864662 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6d103d-443c-4505-856c-e42a667c898c admin admin] PATH_INFO: `/v3/projects` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:47.865260 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6d103d-443c-4505-856c-e42a667c898c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:47.867162 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6d103d-443c-4505-856c-e42a667c898c admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:47.918149 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-d424f0cb-2a67-4c71-a986-f926c2787e50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bd7cee3f5704d01ae9df6d300b30004', 'request_id': 'req-d424f0cb-2a67-4c71-a986-f926c2787e50'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:48.215738 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1047/2066] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:40:47 2022] POST /identity/v3/projects => generated 383 bytes in 369 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:40:48.216644 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6d103d-443c-4505-856c-e42a667c898c admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:48.260754 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-d424f0cb-2a67-4c71-a986-f926c2787e50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bd7cee3f5704d01ae9df6d300b30004', 'request_id': 'req-d424f0cb-2a67-4c71-a986-f926c2787e50'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:48.268785 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-d424f0cb-2a67-4c71-a986-f926c2787e50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bd7cee3f5704d01ae9df6d300b30004', 'request_id': 'req-d424f0cb-2a67-4c71-a986-f926c2787e50'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:48.283204 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6d103d-443c-4505-856c-e42a667c898c admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:48.289688 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab973577-9b8e-477d-a1f7-c5c31ec8895b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:48.289688 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab973577-9b8e-477d-a1f7-c5c31ec8895b admin admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:48.289688 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab973577-9b8e-477d-a1f7-c5c31ec8895b admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:48.290168 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab973577-9b8e-477d-a1f7-c5c31ec8895b admin admin] PATH_INFO: `/v3/projects/19128ae8ce544f70a0fd148c537ca15b/users/970b2ae5af3c499495085e85a51f3c05/roles/3771341b3be641139e752aab690397a4` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:48.293506 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-d424f0cb-2a67-4c71-a986-f926c2787e50 admin admin] The token cache is being invalidate because project 1bd7cee3f5704d01ae9df6d300b30004 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:40:48.294155 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-d424f0cb-2a67-4c71-a986-f926c2787e50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:48.294963 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-d424f0cb-2a67-4c71-a986-f926c2787e50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:48.297386 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1020/2067] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:40:47 2022] DELETE /identity/v3/projects/1bd7cee3f5704d01ae9df6d300b30004 => generated 0 bytes in 576 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:40:48.299277 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d424f0cb-2a67-4c71-a986-f926c2787e50 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:48.302455 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-d424f0cb-2a67-4c71-a986-f926c2787e50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:48.310795 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab973577-9b8e-477d-a1f7-c5c31ec8895b admin admin] RBAC: Authorizing `identity:create_grant(project_id=19128ae8ce544f70a0fd148c537ca15b, user_id=970b2ae5af3c499495085e85a51f3c05, role_id=3771341b3be641139e752aab690397a4)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:48.313711 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab973577-9b8e-477d-a1f7-c5c31ec8895b admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:48.365375 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:40:48.393532 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1048/2068] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:40:48 2022] PUT /identity/v3/projects/19128ae8ce544f70a0fd148c537ca15b/users/970b2ae5af3c499495085e85a51f3c05/roles/3771341b3be641139e752aab690397a4 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:40:48.395080 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f48346-8a41-468b-94fc-8fc163bbd9a8 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:48.395522 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f48346-8a41-468b-94fc-8fc163bbd9a8 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:48.395848 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f48346-8a41-468b-94fc-8fc163bbd9a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:48.418489 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d424f0cb-2a67-4c71-a986-f926c2787e50 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:48.419718 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-31f48346-8a41-468b-94fc-8fc163bbd9a8 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:40:48.421023 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-31f48346-8a41-468b-94fc-8fc163bbd9a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:48.483343 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e36af6-50b6-49b6-baf2-b96b404cb27a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:48.484110 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e36af6-50b6-49b6-baf2-b96b404cb27a admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:48.484435 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e36af6-50b6-49b6-baf2-b96b404cb27a admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:48.484722 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e36af6-50b6-49b6-baf2-b96b404cb27a admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:48.485841 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e36af6-50b6-49b6-baf2-b96b404cb27a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:48.489067 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e36af6-50b6-49b6-baf2-b96b404cb27a admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:48.489222 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1049/2069] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:40:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:40:48.489518 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fcc985-7d79-4dae-971f-cd93c41d48a0 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:48.489859 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fcc985-7d79-4dae-971f-cd93c41d48a0 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:48.490089 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fcc985-7d79-4dae-971f-cd93c41d48a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:48.522976 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-48fcc985-7d79-4dae-971f-cd93c41d48a0 None None] MFA Rules not processed for user `970b2ae5af3c499495085e85a51f3c05`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:40:48.524187 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-48fcc985-7d79-4dae-971f-cd93c41d48a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:48.571438 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1021/2070] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:40:48 2022] POST /identity/v3/users => generated 292 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:40:48.572825 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f6eaad-1298-4c58-9e36-ca5b9dfe23a9 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:48.573116 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f6eaad-1298-4c58-9e36-ca5b9dfe23a9 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:48.573398 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f6eaad-1298-4c58-9e36-ca5b9dfe23a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:48.592101 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-d7f6eaad-1298-4c58-9e36-ca5b9dfe23a9 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:40:48.593334 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-d7f6eaad-1298-4c58-9e36-ca5b9dfe23a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:48.597916 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1050/2071] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:40:48 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:40:48.598754 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fcc985-7d79-4dae-971f-cd93c41d48a0 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:48.604475 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1022/2072] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:40:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:40:48.605848 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f6eaad-1298-4c58-9e36-ca5b9dfe23a9 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:48.614783 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f6eaad-1298-4c58-9e36-ca5b9dfe23a9 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:48.614913 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fcc985-7d79-4dae-971f-cd93c41d48a0 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:48.620017 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21940ec8-1f0a-4b00-9a98-dffc29b1c544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:48.620724 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21940ec8-1f0a-4b00-9a98-dffc29b1c544 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:48.621009 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21940ec8-1f0a-4b00-9a98-dffc29b1c544 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:48.621279 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21940ec8-1f0a-4b00-9a98-dffc29b1c544 admin admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:48.622062 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21940ec8-1f0a-4b00-9a98-dffc29b1c544 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:40:48.622429 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21940ec8-1f0a-4b00-9a98-dffc29b1c544 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:48.623178 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8458801-0cc6-4793-be72-08561563ba1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:48.623866 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8458801-0cc6-4793-be72-08561563ba1d admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:48.624140 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21940ec8-1f0a-4b00-9a98-dffc29b1c544 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:48.624251 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8458801-0cc6-4793-be72-08561563ba1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:48.624467 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8458801-0cc6-4793-be72-08561563ba1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:48.625221 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8458801-0cc6-4793-be72-08561563ba1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:48.626851 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8458801-0cc6-4793-be72-08561563ba1d admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:48.643568 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1051/2073] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:40:48 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:40:48.643568 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8458801-0cc6-4793-be72-08561563ba1d admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:48.643793 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1023/2074] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jun 7 14:40:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:40:48.643894 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-f8458801-0cc6-4793-be72-08561563ba1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:48.644843 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21940ec8-1f0a-4b00-9a98-dffc29b1c544 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:48.656014 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21940ec8-1f0a-4b00-9a98-dffc29b1c544 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:48.661963 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646bcfb3-0c60-454a-9bb8-a6051e48997b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:48.662653 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646bcfb3-0c60-454a-9bb8-a6051e48997b admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:48.662951 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646bcfb3-0c60-454a-9bb8-a6051e48997b admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:48.663240 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646bcfb3-0c60-454a-9bb8-a6051e48997b admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:48.664174 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646bcfb3-0c60-454a-9bb8-a6051e48997b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:48.666013 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646bcfb3-0c60-454a-9bb8-a6051e48997b admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:48.683833 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1024/2075] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:40:48 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 14:40:48.696660 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646bcfb3-0c60-454a-9bb8-a6051e48997b admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:48.698358 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646bcfb3-0c60-454a-9bb8-a6051e48997b admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:48.702986 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bc2a43-3ba5-418e-8f82-8c241e24584e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:48.703772 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bc2a43-3ba5-418e-8f82-8c241e24584e admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:48.704050 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bc2a43-3ba5-418e-8f82-8c241e24584e admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:48.704324 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bc2a43-3ba5-418e-8f82-8c241e24584e admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:48.708012 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bc2a43-3ba5-418e-8f82-8c241e24584e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:48.708012 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bc2a43-3ba5-418e-8f82-8c241e24584e admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:48.708226 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:40:48.726220 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8458801-0cc6-4793-be72-08561563ba1d admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:48.729955 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2881f8-9246-4efb-bd1b-af8169af8cc6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d81f597cb44c56a966537089bcce4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:48.730610 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2881f8-9246-4efb-bd1b-af8169af8cc6 service cinder] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:48.730900 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2881f8-9246-4efb-bd1b-af8169af8cc6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:48.731180 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2881f8-9246-4efb-bd1b-af8169af8cc6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:48.739662 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2881f8-9246-4efb-bd1b-af8169af8cc6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:48.741122 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2881f8-9246-4efb-bd1b-af8169af8cc6 service cinder] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:48.751129 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1052/2076] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:40:48 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:40:48.753693 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2881f8-9246-4efb-bd1b-af8169af8cc6 service cinder] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:48.762662 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2881f8-9246-4efb-bd1b-af8169af8cc6 service cinder] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:48.768834 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea74c27f-8485-44f9-9c00-420a33a2da13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:48.770957 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea74c27f-8485-44f9-9c00-420a33a2da13 admin admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:48.770957 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea74c27f-8485-44f9-9c00-420a33a2da13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:48.770957 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea74c27f-8485-44f9-9c00-420a33a2da13 admin admin] PATH_INFO: `/v3/projects/74031805ea574215b6a9e5e49ebaf802/users/f1549f8782874331a3655ca512b7c71d/roles/3771341b3be641139e752aab690397a4` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:48.786600 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea74c27f-8485-44f9-9c00-420a33a2da13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74031805ea574215b6a9e5e49ebaf802, user_id=f1549f8782874331a3655ca512b7c71d, role_id=3771341b3be641139e752aab690397a4)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:48.789312 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea74c27f-8485-44f9-9c00-420a33a2da13 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:48.859767 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1025/2077] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:40:48 2022] POST /identity/v3/projects => generated 387 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:40:48.860715 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bc2a43-3ba5-418e-8f82-8c241e24584e admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:48.878983 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1053/2078] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:40:48 2022] PUT /identity/v3/projects/74031805ea574215b6a9e5e49ebaf802/users/f1549f8782874331a3655ca512b7c71d/roles/3771341b3be641139e752aab690397a4 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:40:48.890827 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ed93a3-401f-4b24-b524-83bec1d55473 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:48.891805 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ed93a3-401f-4b24-b524-83bec1d55473 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:48.891938 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ed93a3-401f-4b24-b524-83bec1d55473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:48.920602 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-f9ed93a3-401f-4b24-b524-83bec1d55473 None None] MFA Rules not processed for user `f1549f8782874331a3655ca512b7c71d`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:40:48.921801 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-f9ed93a3-401f-4b24-b524-83bec1d55473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:48.940286 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bc2a43-3ba5-418e-8f82-8c241e24584e admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:48.991330 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1054/2079] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:40:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:40:49.003385 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b27632-7e4c-437b-b42a-c508d0b59fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:49.003647 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ed93a3-401f-4b24-b524-83bec1d55473 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:49.004144 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b27632-7e4c-437b-b42a-c508d0b59fd3 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:49.004456 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b27632-7e4c-437b-b42a-c508d0b59fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:49.004729 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b27632-7e4c-437b-b42a-c508d0b59fd3 admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:49.005443 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b27632-7e4c-437b-b42a-c508d0b59fd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:49.005629 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-f9ed93a3-401f-4b24-b524-83bec1d55473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:49.007356 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b27632-7e4c-437b-b42a-c508d0b59fd3 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:49.068112 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ed93a3-401f-4b24-b524-83bec1d55473 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:49.072241 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91909245-e563-45d6-b444-ffa445186b81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d81f597cb44c56a966537089bcce4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:49.072924 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91909245-e563-45d6-b444-ffa445186b81 service cinder] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:49.073221 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91909245-e563-45d6-b444-ffa445186b81 service cinder] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:49.073485 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91909245-e563-45d6-b444-ffa445186b81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:49.082109 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91909245-e563-45d6-b444-ffa445186b81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:49.083532 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91909245-e563-45d6-b444-ffa445186b81 service cinder] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:49.092915 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1055/2080] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:40:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:40:49.095783 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:40:49.114206 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1026/2081] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:40:48 2022] POST /identity/v3/users => generated 294 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:40:49.116234 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b27632-7e4c-437b-b42a-c508d0b59fd3 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:49.116708 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91909245-e563-45d6-b444-ffa445186b81 service cinder] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:49.126524 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b27632-7e4c-437b-b42a-c508d0b59fd3 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:49.130019 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91909245-e563-45d6-b444-ffa445186b81 service cinder] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:49.132089 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb97413-b5b3-44dd-8f76-0734a4b6e9d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:49.132907 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb97413-b5b3-44dd-8f76-0734a4b6e9d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:49.133296 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb97413-b5b3-44dd-8f76-0734a4b6e9d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:49.134285 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb97413-b5b3-44dd-8f76-0734a4b6e9d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:49.134405 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb97413-b5b3-44dd-8f76-0734a4b6e9d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:49.135376 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da00979-d079-4fa0-a187-1c7b5a2c3f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:49.136683 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb97413-b5b3-44dd-8f76-0734a4b6e9d6 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:49.136826 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da00979-d079-4fa0-a187-1c7b5a2c3f98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:49.136826 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da00979-d079-4fa0-a187-1c7b5a2c3f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:49.136982 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da00979-d079-4fa0-a187-1c7b5a2c3f98 admin admin] PATH_INFO: `/v3/users/f1549f8782874331a3655ca512b7c71d` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:49.138188 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da00979-d079-4fa0-a187-1c7b5a2c3f98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1549f8782874331a3655ca512b7c71d)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:49.140731 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da00979-d079-4fa0-a187-1c7b5a2c3f98 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:49.146123 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:40:49.148275 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1027/2082] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:40:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:40:49.151948 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb97413-b5b3-44dd-8f76-0734a4b6e9d6 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:49.162800 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb97413-b5b3-44dd-8f76-0734a4b6e9d6 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:49.167767 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebe2c90-0e4c-44fe-85a1-0300c84c53f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:49.168601 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebe2c90-0e4c-44fe-85a1-0300c84c53f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:49.168953 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebe2c90-0e4c-44fe-85a1-0300c84c53f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:49.169309 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebe2c90-0e4c-44fe-85a1-0300c84c53f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:49.170112 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebe2c90-0e4c-44fe-85a1-0300c84c53f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:49.171783 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebe2c90-0e4c-44fe-85a1-0300c84c53f2 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:49.184841 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1028/2083] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:40:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:40:49.189392 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebe2c90-0e4c-44fe-85a1-0300c84c53f2 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:49.199850 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebe2c90-0e4c-44fe-85a1-0300c84c53f2 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:49.205158 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05f4ff4-91d1-4e45-bb08-327df252fe30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:49.206752 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f4ff4-91d1-4e45-bb08-327df252fe30 admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:49.207052 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f4ff4-91d1-4e45-bb08-327df252fe30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:49.207326 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f4ff4-91d1-4e45-bb08-327df252fe30 admin admin] PATH_INFO: `/v3/projects/6714e8cfaee34ac3af916af5e4bb1031/users/ed3353d3d8a242e1a6b54d31adf97a6d/roles/3771341b3be641139e752aab690397a4` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:49.228107 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05f4ff4-91d1-4e45-bb08-327df252fe30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6714e8cfaee34ac3af916af5e4bb1031, user_id=ed3353d3d8a242e1a6b54d31adf97a6d, role_id=3771341b3be641139e752aab690397a4)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:49.230843 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05f4ff4-91d1-4e45-bb08-327df252fe30 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:49.324824 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-1da00979-d079-4fa0-a187-1c7b5a2c3f98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1549f8782874331a3655ca512b7c71d'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:49.325484 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-1da00979-d079-4fa0-a187-1c7b5a2c3f98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1549f8782874331a3655ca512b7c71d'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:49.326094 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-1da00979-d079-4fa0-a187-1c7b5a2c3f98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1549f8782874331a3655ca512b7c71d'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:49.333682 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-1da00979-d079-4fa0-a187-1c7b5a2c3f98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1549f8782874331a3655ca512b7c71d'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:49.345148 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-1da00979-d079-4fa0-a187-1c7b5a2c3f98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1549f8782874331a3655ca512b7c71d'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:49.405194 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1029/2084] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:40:49 2022] PUT /identity/v3/projects/6714e8cfaee34ac3af916af5e4bb1031/users/ed3353d3d8a242e1a6b54d31adf97a6d/roles/3771341b3be641139e752aab690397a4 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:40:49.410842 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c9c4d-ccef-49f4-adfe-98c05152e64b None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:49.411131 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c9c4d-ccef-49f4-adfe-98c05152e64b None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:49.411406 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c9c4d-ccef-49f4-adfe-98c05152e64b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:49.429532 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-3d2c9c4d-ccef-49f4-adfe-98c05152e64b None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:40:49.431371 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-3d2c9c4d-ccef-49f4-adfe-98c05152e64b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:49.432750 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1056/2085] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:40:49 2022] DELETE /identity/v3/users/f1549f8782874331a3655ca512b7c71d => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:40:49.442250 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4dfab3-bc39-4e33-ae42-2464719cf4ec None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:49.442374 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4dfab3-bc39-4e33-ae42-2464719cf4ec None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:49.443017 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4dfab3-bc39-4e33-ae42-2464719cf4ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:49.465277 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-8e4dfab3-bc39-4e33-ae42-2464719cf4ec None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:40:49.466508 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-8e4dfab3-bc39-4e33-ae42-2464719cf4ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:49.501481 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1030/2086] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:40:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:40:49.511216 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2c9c4d-ccef-49f4-adfe-98c05152e64b None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:49.512651 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-3d2c9c4d-ccef-49f4-adfe-98c05152e64b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:49.527467 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1057/2087] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:40:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:40:49.538462 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4dfab3-bc39-4e33-ae42-2464719cf4ec None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:49.541311 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-8e4dfab3-bc39-4e33-ae42-2464719cf4ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:49.583395 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2c9c4d-ccef-49f4-adfe-98c05152e64b None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:49.583395 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046032cd-6b09-418c-9ceb-759acd3dca0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:49.583395 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046032cd-6b09-418c-9ceb-759acd3dca0f service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:49.583395 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046032cd-6b09-418c-9ceb-759acd3dca0f service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:49.584117 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046032cd-6b09-418c-9ceb-759acd3dca0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:49.593080 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046032cd-6b09-418c-9ceb-759acd3dca0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:49.594525 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046032cd-6b09-418c-9ceb-759acd3dca0f service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:49.600564 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:40:49.604752 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4dfab3-bc39-4e33-ae42-2464719cf4ec None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:49.606529 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1031/2088] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:40:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 14:40:49.611468 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab379e2-c9d4-4e63-a1bf-4c9b2a162035 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:49.612161 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab379e2-c9d4-4e63-a1bf-4c9b2a162035 service neutron] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:49.612438 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab379e2-c9d4-4e63-a1bf-4c9b2a162035 service neutron] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:49.612725 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab379e2-c9d4-4e63-a1bf-4c9b2a162035 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:49.625845 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab379e2-c9d4-4e63-a1bf-4c9b2a162035 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:49.628467 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab379e2-c9d4-4e63-a1bf-4c9b2a162035 service neutron] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:49.641235 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1058/2089] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:40:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 14:40:50.181563 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046032cd-6b09-418c-9ceb-759acd3dca0f service neutron] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:50.183104 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-046032cd-6b09-418c-9ceb-759acd3dca0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:50.199859 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046032cd-6b09-418c-9ceb-759acd3dca0f service neutron] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:50.209577 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966a94be-faf7-4285-89cc-0a62a8fe3314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:50.210136 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966a94be-faf7-4285-89cc-0a62a8fe3314 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:50.210405 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966a94be-faf7-4285-89cc-0a62a8fe3314 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:50.210711 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966a94be-faf7-4285-89cc-0a62a8fe3314 admin admin] PATH_INFO: `/v3/projects/74031805ea574215b6a9e5e49ebaf802` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:50.213285 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966a94be-faf7-4285-89cc-0a62a8fe3314 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74031805ea574215b6a9e5e49ebaf802)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:50.215256 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966a94be-faf7-4285-89cc-0a62a8fe3314 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:50.345959 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-966a94be-faf7-4285-89cc-0a62a8fe3314 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74031805ea574215b6a9e5e49ebaf802', 'request_id': 'req-966a94be-faf7-4285-89cc-0a62a8fe3314'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:50.460364 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-966a94be-faf7-4285-89cc-0a62a8fe3314 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74031805ea574215b6a9e5e49ebaf802', 'request_id': 'req-966a94be-faf7-4285-89cc-0a62a8fe3314'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:50.480734 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-966a94be-faf7-4285-89cc-0a62a8fe3314 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74031805ea574215b6a9e5e49ebaf802', 'request_id': 'req-966a94be-faf7-4285-89cc-0a62a8fe3314'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:50.494089 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-966a94be-faf7-4285-89cc-0a62a8fe3314 admin admin] The token cache is being invalidate because project 74031805ea574215b6a9e5e49ebaf802 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:40:50.494586 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-966a94be-faf7-4285-89cc-0a62a8fe3314 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:50.495264 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-966a94be-faf7-4285-89cc-0a62a8fe3314 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:50.496906 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1032/2090] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:40:50 2022] DELETE /identity/v3/projects/74031805ea574215b6a9e5e49ebaf802 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:40:50.502368 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baeeaac-8e58-488f-a912-ee92e5bcc045 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:50.502899 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baeeaac-8e58-488f-a912-ee92e5bcc045 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:50.502899 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baeeaac-8e58-488f-a912-ee92e5bcc045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:50.523141 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-3baeeaac-8e58-488f-a912-ee92e5bcc045 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:40:50.524349 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-3baeeaac-8e58-488f-a912-ee92e5bcc045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:50.529748 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:40:50.599448 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1059/2091] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:40:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:40:50.602471 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bb3c17-4e42-44ea-8b6f-320b84b33f65 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:50.603150 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bb3c17-4e42-44ea-8b6f-320b84b33f65 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:50.603150 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bb3c17-4e42-44ea-8b6f-320b84b33f65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:50.622930 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-70bb3c17-4e42-44ea-8b6f-320b84b33f65 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:40:50.624195 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-70bb3c17-4e42-44ea-8b6f-320b84b33f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:50.633674 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1033/2092] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:40:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:40:50.640142 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baeeaac-8e58-488f-a912-ee92e5bcc045 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:50.651140 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baeeaac-8e58-488f-a912-ee92e5bcc045 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:50.657302 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2e319b-3c80-4a2e-8c45-6d72e17ddca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:50.658283 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2e319b-3c80-4a2e-8c45-6d72e17ddca7 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:50.658778 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2e319b-3c80-4a2e-8c45-6d72e17ddca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:50.659076 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2e319b-3c80-4a2e-8c45-6d72e17ddca7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:50.659940 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2e319b-3c80-4a2e-8c45-6d72e17ddca7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:40:50.660259 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2e319b-3c80-4a2e-8c45-6d72e17ddca7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:50.662715 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2e319b-3c80-4a2e-8c45-6d72e17ddca7 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:50.675804 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1060/2093] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jun 7 14:40:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:40:50.680839 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bb3c17-4e42-44ea-8b6f-320b84b33f65 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:50.691049 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bb3c17-4e42-44ea-8b6f-320b84b33f65 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:50.696271 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15d3f2b-ffd0-40db-833b-0b43ef72d15d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:50.697266 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15d3f2b-ffd0-40db-833b-0b43ef72d15d admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:50.697553 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15d3f2b-ffd0-40db-833b-0b43ef72d15d admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:50.697834 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15d3f2b-ffd0-40db-833b-0b43ef72d15d admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:50.698524 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15d3f2b-ffd0-40db-833b-0b43ef72d15d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:50.700474 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15d3f2b-ffd0-40db-833b-0b43ef72d15d admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:50.819422 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1034/2094] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:40:50 2022] POST /identity/v3/projects => generated 373 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:40:50.820680 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2e319b-3c80-4a2e-8c45-6d72e17ddca7 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:50.831098 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2e319b-3c80-4a2e-8c45-6d72e17ddca7 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:50.885291 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb209ba-8711-450c-9d84-73066c80195e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:50.885967 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb209ba-8711-450c-9d84-73066c80195e admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:50.886253 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb209ba-8711-450c-9d84-73066c80195e admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:50.886532 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb209ba-8711-450c-9d84-73066c80195e admin admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:50.887222 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb209ba-8711-450c-9d84-73066c80195e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:50.889434 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb209ba-8711-450c-9d84-73066c80195e admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:50.963040 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1061/2095] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:40:50 2022] POST /identity/v3/users => generated 287 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:40:50.967069 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15d3f2b-ffd0-40db-833b-0b43ef72d15d admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:50.977965 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15d3f2b-ffd0-40db-833b-0b43ef72d15d admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:50.982728 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c9e373-e2b6-445e-aa51-2db5075c17fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:50.983390 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c9e373-e2b6-445e-aa51-2db5075c17fd admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:50.983681 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c9e373-e2b6-445e-aa51-2db5075c17fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:50.983947 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c9e373-e2b6-445e-aa51-2db5075c17fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:50.986757 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c9e373-e2b6-445e-aa51-2db5075c17fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:50.986757 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c9e373-e2b6-445e-aa51-2db5075c17fd admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:51.000051 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1035/2096] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:40:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:40:51.006152 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb209ba-8711-450c-9d84-73066c80195e admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:51.014904 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb209ba-8711-450c-9d84-73066c80195e admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:51.020235 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58863790-ef78-4be6-ac1e-9d6ce9aa6419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:51.020940 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58863790-ef78-4be6-ac1e-9d6ce9aa6419 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:51.021231 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58863790-ef78-4be6-ac1e-9d6ce9aa6419 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:51.021512 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58863790-ef78-4be6-ac1e-9d6ce9aa6419 admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:51.022249 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58863790-ef78-4be6-ac1e-9d6ce9aa6419 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:51.023912 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58863790-ef78-4be6-ac1e-9d6ce9aa6419 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:51.038287 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1062/2097] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:40:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:40:51.039426 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c9e373-e2b6-445e-aa51-2db5075c17fd admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:51.051510 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c9e373-e2b6-445e-aa51-2db5075c17fd admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:51.057668 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfbe042-eeda-484f-9bbe-eb67b175083d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:51.059262 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfbe042-eeda-484f-9bbe-eb67b175083d admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:51.059782 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfbe042-eeda-484f-9bbe-eb67b175083d admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:51.060313 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfbe042-eeda-484f-9bbe-eb67b175083d admin admin] PATH_INFO: `/v3/projects/49b5c8d3b0a1490f81165b16f22f129d/users/47b85b0ea3b84693b2a62374587ef41e/roles/3771341b3be641139e752aab690397a4` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:51.082705 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfbe042-eeda-484f-9bbe-eb67b175083d admin admin] RBAC: Authorizing `identity:create_grant(project_id=49b5c8d3b0a1490f81165b16f22f129d, user_id=47b85b0ea3b84693b2a62374587ef41e, role_id=3771341b3be641139e752aab690397a4)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:51.085400 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfbe042-eeda-484f-9bbe-eb67b175083d admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:51.174836 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1036/2098] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:40:51 2022] PUT /identity/v3/projects/49b5c8d3b0a1490f81165b16f22f129d/users/47b85b0ea3b84693b2a62374587ef41e/roles/3771341b3be641139e752aab690397a4 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:40:51.190827 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff04e4a7-f1eb-4e8c-be10-b557857619ca None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:51.191258 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff04e4a7-f1eb-4e8c-be10-b557857619ca None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:51.191637 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff04e4a7-f1eb-4e8c-be10-b557857619ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:51.219884 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-ff04e4a7-f1eb-4e8c-be10-b557857619ca None None] MFA Rules not processed for user `47b85b0ea3b84693b2a62374587ef41e`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:40:51.221170 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-ff04e4a7-f1eb-4e8c-be10-b557857619ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:51.293353 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1063/2099] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:40:51 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:40:51.304400 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfbe042-eeda-484f-9bbe-eb67b175083d admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:51.306205 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-cbfbe042-eeda-484f-9bbe-eb67b175083d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:51.369445 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfbe042-eeda-484f-9bbe-eb67b175083d admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:51.373470 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd016bc-4004-45e4-9ea4-b50ade980768 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d81f597cb44c56a966537089bcce4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:51.374188 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd016bc-4004-45e4-9ea4-b50ade980768 service cinder] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:51.374477 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd016bc-4004-45e4-9ea4-b50ade980768 service cinder] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:51.374782 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd016bc-4004-45e4-9ea4-b50ade980768 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:51.386561 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd016bc-4004-45e4-9ea4-b50ade980768 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:51.388030 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd016bc-4004-45e4-9ea4-b50ade980768 service cinder] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:51.399450 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1037/2100] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:40:51 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:40:51.418272 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff04e4a7-f1eb-4e8c-be10-b557857619ca None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:51.423872 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:40:51.506668 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff04e4a7-f1eb-4e8c-be10-b557857619ca None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:51.511921 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f968cddd-fb79-4c60-8759-50012815b472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:51.512967 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f968cddd-fb79-4c60-8759-50012815b472 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:51.513607 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f968cddd-fb79-4c60-8759-50012815b472 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:51.513987 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f968cddd-fb79-4c60-8759-50012815b472 admin admin] PATH_INFO: `/v3/users/47b85b0ea3b84693b2a62374587ef41e` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:51.515506 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f968cddd-fb79-4c60-8759-50012815b472 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47b85b0ea3b84693b2a62374587ef41e)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:51.518000 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f968cddd-fb79-4c60-8759-50012815b472 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:51.779977 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-f968cddd-fb79-4c60-8759-50012815b472 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47b85b0ea3b84693b2a62374587ef41e'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:51.780735 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-f968cddd-fb79-4c60-8759-50012815b472 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47b85b0ea3b84693b2a62374587ef41e'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:51.781358 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-f968cddd-fb79-4c60-8759-50012815b472 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47b85b0ea3b84693b2a62374587ef41e'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:51.789411 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-f968cddd-fb79-4c60-8759-50012815b472 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47b85b0ea3b84693b2a62374587ef41e'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:51.810343 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-f968cddd-fb79-4c60-8759-50012815b472 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47b85b0ea3b84693b2a62374587ef41e'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:51.970983 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1064/2101] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:40:51 2022] DELETE /identity/v3/users/47b85b0ea3b84693b2a62374587ef41e => generated 0 bytes in 553 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:40:51.975582 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf86003-1f54-4046-92ae-62b90aff60fd None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:51.976051 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf86003-1f54-4046-92ae-62b90aff60fd None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:51.976150 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf86003-1f54-4046-92ae-62b90aff60fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:51.995110 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-edf86003-1f54-4046-92ae-62b90aff60fd None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:40:51.996365 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-edf86003-1f54-4046-92ae-62b90aff60fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:52.016046 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:40:52.091650 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1038/2102] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:40:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:40:52.098616 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f968cddd-fb79-4c60-8759-50012815b472 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:52.100160 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-f968cddd-fb79-4c60-8759-50012815b472 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:52.170075 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f968cddd-fb79-4c60-8759-50012815b472 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:52.182477 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd2e60f-a584-4d40-bf27-eb7fa93dc90e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:52.183268 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd2e60f-a584-4d40-bf27-eb7fa93dc90e service neutron] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:52.183461 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd2e60f-a584-4d40-bf27-eb7fa93dc90e service neutron] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:52.183690 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd2e60f-a584-4d40-bf27-eb7fa93dc90e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:52.197532 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd2e60f-a584-4d40-bf27-eb7fa93dc90e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:52.199291 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd2e60f-a584-4d40-bf27-eb7fa93dc90e service neutron] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:52.247514 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1065/2103] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:40:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:40:52.744266 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf86003-1f54-4046-92ae-62b90aff60fd None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:52.747468 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-edf86003-1f54-4046-92ae-62b90aff60fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:52.761024 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf86003-1f54-4046-92ae-62b90aff60fd None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:52.777959 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb8dd24-3f49-4dd8-82f7-0e623ec67966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:52.780428 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb8dd24-3f49-4dd8-82f7-0e623ec67966 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:52.780944 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb8dd24-3f49-4dd8-82f7-0e623ec67966 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:52.781460 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb8dd24-3f49-4dd8-82f7-0e623ec67966 admin admin] PATH_INFO: `/v3/projects/49b5c8d3b0a1490f81165b16f22f129d` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:52.783108 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb8dd24-3f49-4dd8-82f7-0e623ec67966 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49b5c8d3b0a1490f81165b16f22f129d)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:52.785286 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb8dd24-3f49-4dd8-82f7-0e623ec67966 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:52.942763 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-ceb8dd24-3f49-4dd8-82f7-0e623ec67966 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49b5c8d3b0a1490f81165b16f22f129d', 'request_id': 'req-ceb8dd24-3f49-4dd8-82f7-0e623ec67966'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:53.015060 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-ceb8dd24-3f49-4dd8-82f7-0e623ec67966 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49b5c8d3b0a1490f81165b16f22f129d', 'request_id': 'req-ceb8dd24-3f49-4dd8-82f7-0e623ec67966'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:53.025328 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-ceb8dd24-3f49-4dd8-82f7-0e623ec67966 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49b5c8d3b0a1490f81165b16f22f129d', 'request_id': 'req-ceb8dd24-3f49-4dd8-82f7-0e623ec67966'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:53.040310 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-ceb8dd24-3f49-4dd8-82f7-0e623ec67966 admin admin] The token cache is being invalidate because project 49b5c8d3b0a1490f81165b16f22f129d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:40:53.040757 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-ceb8dd24-3f49-4dd8-82f7-0e623ec67966 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:53.041502 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-ceb8dd24-3f49-4dd8-82f7-0e623ec67966 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:40:53.046635 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1039/2104] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:40:52 2022] DELETE /identity/v3/projects/49b5c8d3b0a1490f81165b16f22f129d => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:40:53.049017 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d21a121-e0a1-47ff-b6e7-abbcdd8f966a None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:53.049467 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d21a121-e0a1-47ff-b6e7-abbcdd8f966a None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:53.049617 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d21a121-e0a1-47ff-b6e7-abbcdd8f966a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:53.073994 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-7d21a121-e0a1-47ff-b6e7-abbcdd8f966a None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:40:53.075276 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-7d21a121-e0a1-47ff-b6e7-abbcdd8f966a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:53.105080 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:40:53.162064 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1066/2105] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:40:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:40:53.170543 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7e0458-8874-41a9-b1bb-1f88da54a749 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:53.170835 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7e0458-8874-41a9-b1bb-1f88da54a749 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:53.171496 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7e0458-8874-41a9-b1bb-1f88da54a749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:53.193861 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-0a7e0458-8874-41a9-b1bb-1f88da54a749 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:40:53.195444 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-0a7e0458-8874-41a9-b1bb-1f88da54a749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:53.209043 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1040/2106] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:40:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:40:53.214583 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d21a121-e0a1-47ff-b6e7-abbcdd8f966a None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:53.225145 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d21a121-e0a1-47ff-b6e7-abbcdd8f966a None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:53.230102 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5be1f8a-5360-4066-ab62-2d268051cdef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:53.230775 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5be1f8a-5360-4066-ab62-2d268051cdef admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:53.231061 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5be1f8a-5360-4066-ab62-2d268051cdef admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:53.231340 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5be1f8a-5360-4066-ab62-2d268051cdef admin admin] PATH_INFO: `/v3/domains` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:53.232073 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5be1f8a-5360-4066-ab62-2d268051cdef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:40:53.232405 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5be1f8a-5360-4066-ab62-2d268051cdef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:53.234123 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5be1f8a-5360-4066-ab62-2d268051cdef admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:53.249539 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1067/2107] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jun 7 14:40:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:40:53.256299 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7e0458-8874-41a9-b1bb-1f88da54a749 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:53.269828 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7e0458-8874-41a9-b1bb-1f88da54a749 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:53.285350 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c37c04-3135-4d9a-b817-88f2ffa60c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:53.286673 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c37c04-3135-4d9a-b817-88f2ffa60c01 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:53.287172 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c37c04-3135-4d9a-b817-88f2ffa60c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:53.287653 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c37c04-3135-4d9a-b817-88f2ffa60c01 admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:53.288959 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c37c04-3135-4d9a-b817-88f2ffa60c01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:53.292931 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c37c04-3135-4d9a-b817-88f2ffa60c01 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:53.404224 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1041/2108] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:40:53 2022] POST /identity/v3/projects => generated 385 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:40:53.406624 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5be1f8a-5360-4066-ab62-2d268051cdef admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:53.422646 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5be1f8a-5360-4066-ab62-2d268051cdef admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:53.508521 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279879d4-60b5-4967-b624-6339f7609d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:53.509316 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279879d4-60b5-4967-b624-6339f7609d2b admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:53.510366 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279879d4-60b5-4967-b624-6339f7609d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:53.510366 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279879d4-60b5-4967-b624-6339f7609d2b admin admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:53.510815 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279879d4-60b5-4967-b624-6339f7609d2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:53.512629 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279879d4-60b5-4967-b624-6339f7609d2b admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:53.607275 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1068/2109] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:40:53 2022] POST /identity/v3/users => generated 293 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:40:53.610715 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c37c04-3135-4d9a-b817-88f2ffa60c01 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:53.622859 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c37c04-3135-4d9a-b817-88f2ffa60c01 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:53.628790 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ecf902-391d-4d71-abb5-1174e1f898a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:53.630208 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ecf902-391d-4d71-abb5-1174e1f898a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:53.630794 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ecf902-391d-4d71-abb5-1174e1f898a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:53.631367 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ecf902-391d-4d71-abb5-1174e1f898a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:53.632525 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ecf902-391d-4d71-abb5-1174e1f898a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:53.635340 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ecf902-391d-4d71-abb5-1174e1f898a1 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:53.655021 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1042/2110] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:40:53 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:40:53.658820 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279879d4-60b5-4967-b624-6339f7609d2b admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:53.670376 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279879d4-60b5-4967-b624-6339f7609d2b admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:53.675276 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2323f1-d2b3-459e-87ab-3ec1feaa0717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:53.676214 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2323f1-d2b3-459e-87ab-3ec1feaa0717 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:53.676533 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2323f1-d2b3-459e-87ab-3ec1feaa0717 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:53.681707 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2323f1-d2b3-459e-87ab-3ec1feaa0717 admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:53.681707 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2323f1-d2b3-459e-87ab-3ec1feaa0717 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:53.681707 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2323f1-d2b3-459e-87ab-3ec1feaa0717 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:53.691679 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1069/2111] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:40:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:40:53.696153 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ecf902-391d-4d71-abb5-1174e1f898a1 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:53.710952 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ecf902-391d-4d71-abb5-1174e1f898a1 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:53.718189 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd68eeb-f510-4620-8021-de53d8e2684b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:53.721039 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd68eeb-f510-4620-8021-de53d8e2684b admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:53.721539 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd68eeb-f510-4620-8021-de53d8e2684b admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:53.722126 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd68eeb-f510-4620-8021-de53d8e2684b admin admin] PATH_INFO: `/v3/projects/ae68217f6f2940dbb2fb811dd9374429/users/9eaf1051923d4e8f8e4d9956a48a24d6/roles/3771341b3be641139e752aab690397a4` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:53.751489 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd68eeb-f510-4620-8021-de53d8e2684b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae68217f6f2940dbb2fb811dd9374429, user_id=9eaf1051923d4e8f8e4d9956a48a24d6, role_id=3771341b3be641139e752aab690397a4)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:53.754323 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd68eeb-f510-4620-8021-de53d8e2684b admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:53.850533 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1043/2112] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:40:53 2022] PUT /identity/v3/projects/ae68217f6f2940dbb2fb811dd9374429/users/9eaf1051923d4e8f8e4d9956a48a24d6/roles/3771341b3be641139e752aab690397a4 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:40:53.861328 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a4f394-b31f-4241-a406-6263d2ce769f None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:53.861706 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a4f394-b31f-4241-a406-6263d2ce769f None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:53.862069 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a4f394-b31f-4241-a406-6263d2ce769f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:53.916278 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-77a4f394-b31f-4241-a406-6263d2ce769f None None] MFA Rules not processed for user `9eaf1051923d4e8f8e4d9956a48a24d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:40:53.930109 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-77a4f394-b31f-4241-a406-6263d2ce769f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:54.047265 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1070/2113] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:40:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:40:54.058591 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd68eeb-f510-4620-8021-de53d8e2684b admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:54.061846 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-5dd68eeb-f510-4620-8021-de53d8e2684b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:54.122154 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:40:54.135266 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd68eeb-f510-4620-8021-de53d8e2684b admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:54.139254 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f4a0b6-18d6-4a19-9e65-992301f8859d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d81f597cb44c56a966537089bcce4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:54.142877 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f4a0b6-18d6-4a19-9e65-992301f8859d service cinder] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:54.142877 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f4a0b6-18d6-4a19-9e65-992301f8859d service cinder] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:54.142877 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f4a0b6-18d6-4a19-9e65-992301f8859d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:54.150852 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f4a0b6-18d6-4a19-9e65-992301f8859d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:54.152426 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f4a0b6-18d6-4a19-9e65-992301f8859d service cinder] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:54.162617 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1044/2114] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:40:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:40:55.950013 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef091e6b-20ff-4901-92cf-82682af0382e None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:55.950482 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef091e6b-20ff-4901-92cf-82682af0382e None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:55.950605 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef091e6b-20ff-4901-92cf-82682af0382e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:55.996843 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-ef091e6b-20ff-4901-92cf-82682af0382e None None] MFA Rules not processed for user `ed3353d3d8a242e1a6b54d31adf97a6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:40:55.998316 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-ef091e6b-20ff-4901-92cf-82682af0382e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:56.015577 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:40:56.073666 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1071/2115] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:40:55 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:40:56.088902 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f4a0b6-18d6-4a19-9e65-992301f8859d service cinder] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:40:56.090780 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-14f4a0b6-18d6-4a19-9e65-992301f8859d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:56.180863 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f4a0b6-18d6-4a19-9e65-992301f8859d service cinder] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:40:56.186489 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb83f9b6-511a-4bbe-a8ce-20824b90a642 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e857db4902e42989d0d040bbc8517c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:40:56.187142 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb83f9b6-511a-4bbe-a8ce-20824b90a642 service nova] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:56.187427 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb83f9b6-511a-4bbe-a8ce-20824b90a642 service nova] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:56.187704 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb83f9b6-511a-4bbe-a8ce-20824b90a642 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:56.196425 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb83f9b6-511a-4bbe-a8ce-20824b90a642 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:40:56.198083 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb83f9b6-511a-4bbe-a8ce-20824b90a642 service nova] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:40:56.207857 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1045/2116] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:40:56 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:40:58.856125 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15da384-0e30-43d7-a548-64108c7fe94a None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:40:58.856646 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15da384-0e30-43d7-a548-64108c7fe94a None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:40:58.858251 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15da384-0e30-43d7-a548-64108c7fe94a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:40:58.876261 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-a15da384-0e30-43d7-a548-64108c7fe94a None None] MFA Rules not processed for user `ed3353d3d8a242e1a6b54d31adf97a6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:40:58.877756 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-a15da384-0e30-43d7-a548-64108c7fe94a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:40:58.885828 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1072/2117] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:40:58 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:41:06.203135 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=41 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:41:08.870006 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=25 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:41:18.133499 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb83f9b6-511a-4bbe-a8ce-20824b90a642 service nova] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:18.134938 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-bb83f9b6-511a-4bbe-a8ce-20824b90a642 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:41:18.278877 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb83f9b6-511a-4bbe-a8ce-20824b90a642 service nova] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:18.283591 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425177b0-8e56-4d9b-8a61-744f22c65abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:18.284512 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425177b0-8e56-4d9b-8a61-744f22c65abb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:18.284798 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425177b0-8e56-4d9b-8a61-744f22c65abb admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:18.285075 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425177b0-8e56-4d9b-8a61-744f22c65abb admin admin] PATH_INFO: `/v3/users/e3a06e1b6985429d98dcf9aa0c88e741` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:18.286273 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425177b0-8e56-4d9b-8a61-744f22c65abb admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3a06e1b6985429d98dcf9aa0c88e741)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:18.288194 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425177b0-8e56-4d9b-8a61-744f22c65abb admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:18.463599 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-425177b0-8e56-4d9b-8a61-744f22c65abb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3a06e1b6985429d98dcf9aa0c88e741'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:18.464586 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-425177b0-8e56-4d9b-8a61-744f22c65abb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3a06e1b6985429d98dcf9aa0c88e741'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:18.465312 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-425177b0-8e56-4d9b-8a61-744f22c65abb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3a06e1b6985429d98dcf9aa0c88e741'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:18.474385 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-425177b0-8e56-4d9b-8a61-744f22c65abb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3a06e1b6985429d98dcf9aa0c88e741'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:18.491957 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-425177b0-8e56-4d9b-8a61-744f22c65abb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3a06e1b6985429d98dcf9aa0c88e741'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:18.552070 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1046/2118] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:41:18 2022] DELETE /identity/v3/users/e3a06e1b6985429d98dcf9aa0c88e741 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:41:18.556705 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16c0153-8239-4939-a735-64f9f79e5257 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:18.557167 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16c0153-8239-4939-a735-64f9f79e5257 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:18.557297 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16c0153-8239-4939-a735-64f9f79e5257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:18.584394 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-e16c0153-8239-4939-a735-64f9f79e5257 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:41:18.585624 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-e16c0153-8239-4939-a735-64f9f79e5257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:41:18.648014 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1073/2119] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:41:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:41:18.662767 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425177b0-8e56-4d9b-8a61-744f22c65abb admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:18.664242 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-425177b0-8e56-4d9b-8a61-744f22c65abb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:41:18.693783 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:41:18.796423 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425177b0-8e56-4d9b-8a61-744f22c65abb admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:18.800931 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e5803d-7ba5-470e-961f-c029f8daf2f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:18.801755 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e5803d-7ba5-470e-961f-c029f8daf2f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:18.802044 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e5803d-7ba5-470e-961f-c029f8daf2f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:18.802321 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e5803d-7ba5-470e-961f-c029f8daf2f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:18.811578 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e5803d-7ba5-470e-961f-c029f8daf2f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:18.813091 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e5803d-7ba5-470e-961f-c029f8daf2f3 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:18.823469 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1047/2120] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:41:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:41:19.428476 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16c0153-8239-4939-a735-64f9f79e5257 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:19.430032 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-e16c0153-8239-4939-a735-64f9f79e5257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:41:19.444605 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16c0153-8239-4939-a735-64f9f79e5257 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:19.449273 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdab5ef-eef4-4ec4-98c3-963000925cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:19.450280 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdab5ef-eef4-4ec4-98c3-963000925cc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:19.450605 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdab5ef-eef4-4ec4-98c3-963000925cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:19.451055 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdab5ef-eef4-4ec4-98c3-963000925cc5 admin admin] PATH_INFO: `/v3/projects/05644c2217174db280f9b50d8b4628bc` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:19.452523 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdab5ef-eef4-4ec4-98c3-963000925cc5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05644c2217174db280f9b50d8b4628bc)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:19.455837 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdab5ef-eef4-4ec4-98c3-963000925cc5 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:19.553379 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-9cdab5ef-eef4-4ec4-98c3-963000925cc5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05644c2217174db280f9b50d8b4628bc', 'request_id': 'req-9cdab5ef-eef4-4ec4-98c3-963000925cc5'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:19.643337 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-9cdab5ef-eef4-4ec4-98c3-963000925cc5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05644c2217174db280f9b50d8b4628bc', 'request_id': 'req-9cdab5ef-eef4-4ec4-98c3-963000925cc5'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:19.651002 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-9cdab5ef-eef4-4ec4-98c3-963000925cc5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05644c2217174db280f9b50d8b4628bc', 'request_id': 'req-9cdab5ef-eef4-4ec4-98c3-963000925cc5'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:19.662543 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-9cdab5ef-eef4-4ec4-98c3-963000925cc5 admin admin] The token cache is being invalidate because project 05644c2217174db280f9b50d8b4628bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:41:19.662827 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-9cdab5ef-eef4-4ec4-98c3-963000925cc5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:19.663515 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-9cdab5ef-eef4-4ec4-98c3-963000925cc5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:19.665094 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1074/2121] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:41:19 2022] DELETE /identity/v3/projects/05644c2217174db280f9b50d8b4628bc => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:41:19.669568 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e5803d-7ba5-470e-961f-c029f8daf2f3 service neutron] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:19.679972 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e5803d-7ba5-470e-961f-c029f8daf2f3 service neutron] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:19.741520 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3984b5da-a494-482b-b1cf-9e7237c57b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:19.742392 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3984b5da-a494-482b-b1cf-9e7237c57b97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:19.742686 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3984b5da-a494-482b-b1cf-9e7237c57b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:19.742998 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3984b5da-a494-482b-b1cf-9e7237c57b97 admin admin] PATH_INFO: `/v3/users/65b0fd324e044e018311ccb2eed9262f` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:19.744214 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3984b5da-a494-482b-b1cf-9e7237c57b97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65b0fd324e044e018311ccb2eed9262f)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:19.746169 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3984b5da-a494-482b-b1cf-9e7237c57b97 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:19.982200 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-3984b5da-a494-482b-b1cf-9e7237c57b97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65b0fd324e044e018311ccb2eed9262f'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:19.983006 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-3984b5da-a494-482b-b1cf-9e7237c57b97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65b0fd324e044e018311ccb2eed9262f'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:19.983629 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-3984b5da-a494-482b-b1cf-9e7237c57b97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65b0fd324e044e018311ccb2eed9262f'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:19.991130 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-3984b5da-a494-482b-b1cf-9e7237c57b97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65b0fd324e044e018311ccb2eed9262f'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:20.002800 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-3984b5da-a494-482b-b1cf-9e7237c57b97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65b0fd324e044e018311ccb2eed9262f'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:20.056257 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdab5ef-eef4-4ec4-98c3-963000925cc5 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:20.060472 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-9cdab5ef-eef4-4ec4-98c3-963000925cc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:41:20.073010 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1048/2122] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:41:19 2022] DELETE /identity/v3/users/65b0fd324e044e018311ccb2eed9262f => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:41:20.074834 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdab5ef-eef4-4ec4-98c3-963000925cc5 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:20.134729 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46717909-194d-4be5-b5d8-7a244c10fae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:20.135615 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46717909-194d-4be5-b5d8-7a244c10fae5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:20.135905 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46717909-194d-4be5-b5d8-7a244c10fae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:20.136196 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46717909-194d-4be5-b5d8-7a244c10fae5 admin admin] PATH_INFO: `/v3/users/122c179007b443439f84a0fe28868214` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:20.137718 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46717909-194d-4be5-b5d8-7a244c10fae5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=122c179007b443439f84a0fe28868214)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:20.140164 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46717909-194d-4be5-b5d8-7a244c10fae5 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:20.172250 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:41:20.210656 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3984b5da-a494-482b-b1cf-9e7237c57b97 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:20.214594 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-3984b5da-a494-482b-b1cf-9e7237c57b97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:41:20.222003 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:41:20.226487 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3984b5da-a494-482b-b1cf-9e7237c57b97 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:20.231595 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138e5b63-110c-473a-9c93-f41244fc313d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:20.232546 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138e5b63-110c-473a-9c93-f41244fc313d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:20.232829 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138e5b63-110c-473a-9c93-f41244fc313d admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:20.233101 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138e5b63-110c-473a-9c93-f41244fc313d admin admin] PATH_INFO: `/v3/users/9eaf1051923d4e8f8e4d9956a48a24d6` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:20.234297 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138e5b63-110c-473a-9c93-f41244fc313d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9eaf1051923d4e8f8e4d9956a48a24d6)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:20.236270 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138e5b63-110c-473a-9c93-f41244fc313d admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:20.337733 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-46717909-194d-4be5-b5d8-7a244c10fae5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '122c179007b443439f84a0fe28868214'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:20.339965 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-46717909-194d-4be5-b5d8-7a244c10fae5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '122c179007b443439f84a0fe28868214'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:20.340643 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-46717909-194d-4be5-b5d8-7a244c10fae5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '122c179007b443439f84a0fe28868214'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:20.350094 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-46717909-194d-4be5-b5d8-7a244c10fae5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '122c179007b443439f84a0fe28868214'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:20.364817 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-46717909-194d-4be5-b5d8-7a244c10fae5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '122c179007b443439f84a0fe28868214'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:20.438255 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1075/2123] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:41:20 2022] DELETE /identity/v3/users/122c179007b443439f84a0fe28868214 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:41:20.443804 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cfa5d9-f6bc-481c-9462-3a4957878f1b None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:20.444333 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cfa5d9-f6bc-481c-9462-3a4957878f1b None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:20.444815 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cfa5d9-f6bc-481c-9462-3a4957878f1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:20.451128 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-138e5b63-110c-473a-9c93-f41244fc313d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9eaf1051923d4e8f8e4d9956a48a24d6'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:20.452045 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-138e5b63-110c-473a-9c93-f41244fc313d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9eaf1051923d4e8f8e4d9956a48a24d6'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:20.452781 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-138e5b63-110c-473a-9c93-f41244fc313d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9eaf1051923d4e8f8e4d9956a48a24d6'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:20.460624 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-138e5b63-110c-473a-9c93-f41244fc313d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9eaf1051923d4e8f8e4d9956a48a24d6'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:20.465972 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-16cfa5d9-f6bc-481c-9462-3a4957878f1b None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:41:20.467252 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-16cfa5d9-f6bc-481c-9462-3a4957878f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:41:20.473242 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-138e5b63-110c-473a-9c93-f41244fc313d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9eaf1051923d4e8f8e4d9956a48a24d6'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:20.537819 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1076/2124] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:41:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:41:20.538902 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cfa5d9-f6bc-481c-9462-3a4957878f1b None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:20.541040 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-16cfa5d9-f6bc-481c-9462-3a4957878f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:41:20.556155 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cfa5d9-f6bc-481c-9462-3a4957878f1b None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:20.562177 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528bba4e-acc3-4731-9705-fddb5fb3326c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:20.563185 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528bba4e-acc3-4731-9705-fddb5fb3326c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:20.563542 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528bba4e-acc3-4731-9705-fddb5fb3326c admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:20.563893 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528bba4e-acc3-4731-9705-fddb5fb3326c admin admin] PATH_INFO: `/v3/projects/a793d314b8a544bb9ad12d7474f0e22a` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:20.565404 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528bba4e-acc3-4731-9705-fddb5fb3326c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a793d314b8a544bb9ad12d7474f0e22a)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:20.567429 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528bba4e-acc3-4731-9705-fddb5fb3326c admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:20.582331 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1049/2125] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:41:20 2022] DELETE /identity/v3/users/9eaf1051923d4e8f8e4d9956a48a24d6 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:41:20.582331 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138e5b63-110c-473a-9c93-f41244fc313d admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:20.583429 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-138e5b63-110c-473a-9c93-f41244fc313d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:41:20.656362 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138e5b63-110c-473a-9c93-f41244fc313d admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:20.661422 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7328c697-7b7b-4589-85c3-e49f8cf55fb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:20.661743 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7328c697-7b7b-4589-85c3-e49f8cf55fb8 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:20.662027 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7328c697-7b7b-4589-85c3-e49f8cf55fb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:20.662302 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7328c697-7b7b-4589-85c3-e49f8cf55fb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:20.688737 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-528bba4e-acc3-4731-9705-fddb5fb3326c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a793d314b8a544bb9ad12d7474f0e22a', 'request_id': 'req-528bba4e-acc3-4731-9705-fddb5fb3326c'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:20.733850 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7328c697-7b7b-4589-85c3-e49f8cf55fb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:20.735300 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7328c697-7b7b-4589-85c3-e49f8cf55fb8 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:20.745636 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1050/2126] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:41:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:41:20.747038 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d2c079-f809-4d75-97f8-5c3207509b4c None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:20.747332 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d2c079-f809-4d75-97f8-5c3207509b4c None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:20.747657 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d2c079-f809-4d75-97f8-5c3207509b4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:20.750983 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-528bba4e-acc3-4731-9705-fddb5fb3326c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a793d314b8a544bb9ad12d7474f0e22a', 'request_id': 'req-528bba4e-acc3-4731-9705-fddb5fb3326c'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:20.758364 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-528bba4e-acc3-4731-9705-fddb5fb3326c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a793d314b8a544bb9ad12d7474f0e22a', 'request_id': 'req-528bba4e-acc3-4731-9705-fddb5fb3326c'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:20.768624 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-56d2c079-f809-4d75-97f8-5c3207509b4c None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:41:20.775518 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-56d2c079-f809-4d75-97f8-5c3207509b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:41:20.775739 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-528bba4e-acc3-4731-9705-fddb5fb3326c admin admin] The token cache is being invalidate because project a793d314b8a544bb9ad12d7474f0e22a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:41:20.775739 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-528bba4e-acc3-4731-9705-fddb5fb3326c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:20.775739 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-528bba4e-acc3-4731-9705-fddb5fb3326c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:20.775739 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1077/2127] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:41:20 2022] DELETE /identity/v3/projects/a793d314b8a544bb9ad12d7474f0e22a => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:41:20.781297 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6db132-f43c-4dc7-b430-3f01d94c9e4a None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:20.781597 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6db132-f43c-4dc7-b430-3f01d94c9e4a None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:20.782262 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6db132-f43c-4dc7-b430-3f01d94c9e4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:20.814362 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:41:20.827984 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-6e6db132-f43c-4dc7-b430-3f01d94c9e4a None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:41:20.829258 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-6e6db132-f43c-4dc7-b430-3f01d94c9e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:41:20.855962 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1051/2128] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:41:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:41:20.855962 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:41:20.867978 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d2c079-f809-4d75-97f8-5c3207509b4c None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:20.869539 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-56d2c079-f809-4d75-97f8-5c3207509b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:41:20.901352 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1078/2129] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:41:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:41:20.909304 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79de96a2-fdf5-4f45-ae21-8d2a236297bb None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:20.909540 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79de96a2-fdf5-4f45-ae21-8d2a236297bb None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:20.909848 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79de96a2-fdf5-4f45-ae21-8d2a236297bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:20.931612 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d2c079-f809-4d75-97f8-5c3207509b4c None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:20.932034 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-79de96a2-fdf5-4f45-ae21-8d2a236297bb None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:41:20.933449 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-79de96a2-fdf5-4f45-ae21-8d2a236297bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:41:20.935694 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e672215-64fc-4918-a2ff-69fef61bb947 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:20.936923 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e672215-64fc-4918-a2ff-69fef61bb947 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:20.936923 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e672215-64fc-4918-a2ff-69fef61bb947 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:20.936923 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e672215-64fc-4918-a2ff-69fef61bb947 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:20.952535 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1079/2130] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:41:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:41:20.965560 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79de96a2-fdf5-4f45-ae21-8d2a236297bb None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:20.976612 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e672215-64fc-4918-a2ff-69fef61bb947 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:20.980788 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e672215-64fc-4918-a2ff-69fef61bb947 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:20.982298 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79de96a2-fdf5-4f45-ae21-8d2a236297bb None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:20.988341 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e9fa68-39d5-4177-a6b6-96fa0d64bc60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:20.993679 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e9fa68-39d5-4177-a6b6-96fa0d64bc60 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:20.993679 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e9fa68-39d5-4177-a6b6-96fa0d64bc60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:20.993679 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e9fa68-39d5-4177-a6b6-96fa0d64bc60 admin admin] PATH_INFO: `/v3/domains` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:20.993679 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e9fa68-39d5-4177-a6b6-96fa0d64bc60 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:41:20.993679 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e9fa68-39d5-4177-a6b6-96fa0d64bc60 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:20.993679 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e9fa68-39d5-4177-a6b6-96fa0d64bc60 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:21.004255 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1052/2131] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:41:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 14:41:21.017349 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1080/2132] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jun 7 14:41:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:41:21.021815 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e672215-64fc-4918-a2ff-69fef61bb947 service neutron] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:21.035472 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e672215-64fc-4918-a2ff-69fef61bb947 service neutron] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:21.041368 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad15fa32-0fdb-4b38-a47f-e498ad3d7fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:21.042285 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15fa32-0fdb-4b38-a47f-e498ad3d7fb9 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:21.042572 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15fa32-0fdb-4b38-a47f-e498ad3d7fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:21.042888 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15fa32-0fdb-4b38-a47f-e498ad3d7fb9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:21.043541 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad15fa32-0fdb-4b38-a47f-e498ad3d7fb9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:21.045495 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad15fa32-0fdb-4b38-a47f-e498ad3d7fb9 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:21.142083 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1053/2133] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:41:21 2022] POST /identity/v3/projects => generated 369 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:41:21.151282 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e9fa68-39d5-4177-a6b6-96fa0d64bc60 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:21.158687 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e9fa68-39d5-4177-a6b6-96fa0d64bc60 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:21.220639 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b727e0-344d-429f-b07c-6d4a3d5d5bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:21.221682 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b727e0-344d-429f-b07c-6d4a3d5d5bfe admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:21.221682 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b727e0-344d-429f-b07c-6d4a3d5d5bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:21.221868 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b727e0-344d-429f-b07c-6d4a3d5d5bfe admin admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:21.222549 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b727e0-344d-429f-b07c-6d4a3d5d5bfe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:21.224538 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b727e0-344d-429f-b07c-6d4a3d5d5bfe admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:21.324459 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1081/2134] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:41:21 2022] POST /identity/v3/users => generated 285 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:41:21.328724 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad15fa32-0fdb-4b38-a47f-e498ad3d7fb9 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:21.342354 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad15fa32-0fdb-4b38-a47f-e498ad3d7fb9 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:21.348433 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb07a0d7-ae8b-477b-a45b-caa2b0bf596e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:21.354021 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb07a0d7-ae8b-477b-a45b-caa2b0bf596e admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:21.354021 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb07a0d7-ae8b-477b-a45b-caa2b0bf596e admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:21.354021 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb07a0d7-ae8b-477b-a45b-caa2b0bf596e admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:21.354021 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb07a0d7-ae8b-477b-a45b-caa2b0bf596e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:21.354021 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb07a0d7-ae8b-477b-a45b-caa2b0bf596e admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:21.365015 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1054/2135] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:41:21 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:41:21.370791 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b727e0-344d-429f-b07c-6d4a3d5d5bfe admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:21.386968 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b727e0-344d-429f-b07c-6d4a3d5d5bfe admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:21.391931 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21d68f0-a022-4d40-9bf7-d2c36139ad90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:21.392629 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21d68f0-a022-4d40-9bf7-d2c36139ad90 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:21.392915 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21d68f0-a022-4d40-9bf7-d2c36139ad90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:21.393192 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21d68f0-a022-4d40-9bf7-d2c36139ad90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:21.394577 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21d68f0-a022-4d40-9bf7-d2c36139ad90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:21.396165 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21d68f0-a022-4d40-9bf7-d2c36139ad90 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:21.406658 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1082/2136] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:41:21 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:41:21.411513 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb07a0d7-ae8b-477b-a45b-caa2b0bf596e admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:21.418930 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21d68f0-a022-4d40-9bf7-d2c36139ad90 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:21.421021 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-e21d68f0-a022-4d40-9bf7-d2c36139ad90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:41:21.422271 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb07a0d7-ae8b-477b-a45b-caa2b0bf596e admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:21.433712 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb700492-b1c8-4fff-86b7-abe53ac54f84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:21.433972 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb700492-b1c8-4fff-86b7-abe53ac54f84 admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:21.434182 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb700492-b1c8-4fff-86b7-abe53ac54f84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:21.435334 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb700492-b1c8-4fff-86b7-abe53ac54f84 admin admin] PATH_INFO: `/v3/projects/a95d925b241c477db7f0e8ca7fdef3ee/users/4c2185768fe04873a227855201b7234d/roles/3771341b3be641139e752aab690397a4` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:21.449169 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21d68f0-a022-4d40-9bf7-d2c36139ad90 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:21.455347 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56b5bac-1e5b-456a-803d-392a9eacfaa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:21.456882 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56b5bac-1e5b-456a-803d-392a9eacfaa5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:21.457516 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56b5bac-1e5b-456a-803d-392a9eacfaa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:21.457713 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56b5bac-1e5b-456a-803d-392a9eacfaa5 admin admin] PATH_INFO: `/v3/projects/40a2691996124d37a5a292f7ea4a2f78` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:21.459131 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56b5bac-1e5b-456a-803d-392a9eacfaa5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40a2691996124d37a5a292f7ea4a2f78)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:21.461645 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56b5bac-1e5b-456a-803d-392a9eacfaa5 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:21.472502 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb700492-b1c8-4fff-86b7-abe53ac54f84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a95d925b241c477db7f0e8ca7fdef3ee, user_id=4c2185768fe04873a227855201b7234d, role_id=3771341b3be641139e752aab690397a4)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:21.486167 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb700492-b1c8-4fff-86b7-abe53ac54f84 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:21.564868 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1055/2137] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:41:21 2022] PUT /identity/v3/projects/a95d925b241c477db7f0e8ca7fdef3ee/users/4c2185768fe04873a227855201b7234d/roles/3771341b3be641139e752aab690397a4 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:41:21.575920 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274dc098-8bdd-4f2a-9fd4-3c3e5b15c5e9 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:21.576214 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274dc098-8bdd-4f2a-9fd4-3c3e5b15c5e9 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:21.576537 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274dc098-8bdd-4f2a-9fd4-3c3e5b15c5e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:21.597947 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-f56b5bac-1e5b-456a-803d-392a9eacfaa5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40a2691996124d37a5a292f7ea4a2f78', 'request_id': 'req-f56b5bac-1e5b-456a-803d-392a9eacfaa5'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:21.604798 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-274dc098-8bdd-4f2a-9fd4-3c3e5b15c5e9 None None] MFA Rules not processed for user `4c2185768fe04873a227855201b7234d`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:41:21.606042 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-274dc098-8bdd-4f2a-9fd4-3c3e5b15c5e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:41:21.666434 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-f56b5bac-1e5b-456a-803d-392a9eacfaa5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40a2691996124d37a5a292f7ea4a2f78', 'request_id': 'req-f56b5bac-1e5b-456a-803d-392a9eacfaa5'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:21.675979 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=90,keystone:INSERT=4,keystone:DELETE=6 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:41:21.675979 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-f56b5bac-1e5b-456a-803d-392a9eacfaa5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40a2691996124d37a5a292f7ea4a2f78', 'request_id': 'req-f56b5bac-1e5b-456a-803d-392a9eacfaa5'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:21.682047 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1056/2138] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:41:21 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:41:21.683068 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274dc098-8bdd-4f2a-9fd4-3c3e5b15c5e9 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:21.684461 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-274dc098-8bdd-4f2a-9fd4-3c3e5b15c5e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:41:21.689426 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:41:21.689709 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-f56b5bac-1e5b-456a-803d-392a9eacfaa5 admin admin] The token cache is being invalidate because project 40a2691996124d37a5a292f7ea4a2f78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:41:21.690174 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-f56b5bac-1e5b-456a-803d-392a9eacfaa5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:21.691366 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-f56b5bac-1e5b-456a-803d-392a9eacfaa5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:21.696320 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1083/2139] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:41:21 2022] DELETE /identity/v3/projects/40a2691996124d37a5a292f7ea4a2f78 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:41:21.696320 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56b5bac-1e5b-456a-803d-392a9eacfaa5 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:21.696320 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-f56b5bac-1e5b-456a-803d-392a9eacfaa5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:41:21.762019 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274dc098-8bdd-4f2a-9fd4-3c3e5b15c5e9 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:21.768292 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826c1a10-6340-4916-991e-a50699354c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:21.769272 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826c1a10-6340-4916-991e-a50699354c51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:21.769546 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826c1a10-6340-4916-991e-a50699354c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:21.770009 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826c1a10-6340-4916-991e-a50699354c51 admin admin] PATH_INFO: `/v3/projects/ae68217f6f2940dbb2fb811dd9374429` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:21.771356 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826c1a10-6340-4916-991e-a50699354c51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae68217f6f2940dbb2fb811dd9374429)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:21.772518 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56b5bac-1e5b-456a-803d-392a9eacfaa5 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:21.773522 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826c1a10-6340-4916-991e-a50699354c51 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:21.779523 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a48cb0-c261-42e8-8ecb-92a44c30b55b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:21.780324 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a48cb0-c261-42e8-8ecb-92a44c30b55b admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:21.780741 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a48cb0-c261-42e8-8ecb-92a44c30b55b admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:21.781166 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a48cb0-c261-42e8-8ecb-92a44c30b55b admin admin] PATH_INFO: `/v3/projects` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:21.781955 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a48cb0-c261-42e8-8ecb-92a44c30b55b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:21.784144 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a48cb0-c261-42e8-8ecb-92a44c30b55b admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:21.964444 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1084/2140] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:41:21 2022] POST /identity/v3/projects => generated 367 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:41:21.965764 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c601ab2-17b3-4a7e-919d-1ab641d5bcf5 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:21.966053 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c601ab2-17b3-4a7e-919d-1ab641d5bcf5 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:21.966330 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c601ab2-17b3-4a7e-919d-1ab641d5bcf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:21.990676 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-6c601ab2-17b3-4a7e-919d-1ab641d5bcf5 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:41:21.991902 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-6c601ab2-17b3-4a7e-919d-1ab641d5bcf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:41:21.999953 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-826c1a10-6340-4916-991e-a50699354c51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae68217f6f2940dbb2fb811dd9374429', 'request_id': 'req-826c1a10-6340-4916-991e-a50699354c51'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:22.050794 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1085/2141] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:41:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:41:22.051695 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c601ab2-17b3-4a7e-919d-1ab641d5bcf5 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:22.062208 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c601ab2-17b3-4a7e-919d-1ab641d5bcf5 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:22.066983 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64e97b6-d300-4d28-a892-835c8ceac0e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:22.067632 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64e97b6-d300-4d28-a892-835c8ceac0e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:22.068149 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64e97b6-d300-4d28-a892-835c8ceac0e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:22.068278 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64e97b6-d300-4d28-a892-835c8ceac0e2 admin admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:22.068899 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64e97b6-d300-4d28-a892-835c8ceac0e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:22.070843 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64e97b6-d300-4d28-a892-835c8ceac0e2 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:22.071139 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-826c1a10-6340-4916-991e-a50699354c51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae68217f6f2940dbb2fb811dd9374429', 'request_id': 'req-826c1a10-6340-4916-991e-a50699354c51'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:22.077867 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-826c1a10-6340-4916-991e-a50699354c51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae68217f6f2940dbb2fb811dd9374429', 'request_id': 'req-826c1a10-6340-4916-991e-a50699354c51'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:22.089554 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-826c1a10-6340-4916-991e-a50699354c51 admin admin] The token cache is being invalidate because project ae68217f6f2940dbb2fb811dd9374429 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:41:22.089845 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-826c1a10-6340-4916-991e-a50699354c51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:22.090507 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-826c1a10-6340-4916-991e-a50699354c51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:22.092043 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1057/2142] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:41:21 2022] DELETE /identity/v3/projects/ae68217f6f2940dbb2fb811dd9374429 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:41:22.096527 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28077740-5d7a-4fe7-acc1-a538f0b9b490 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:22.097324 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28077740-5d7a-4fe7-acc1-a538f0b9b490 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:22.097606 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28077740-5d7a-4fe7-acc1-a538f0b9b490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:22.116237 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-28077740-5d7a-4fe7-acc1-a538f0b9b490 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:41:22.117485 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-28077740-5d7a-4fe7-acc1-a538f0b9b490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:41:22.125504 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1058/2143] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:41:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:41:22.126799 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd17c73-8962-4cab-88c8-5f159e8c82d5 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:22.127057 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd17c73-8962-4cab-88c8-5f159e8c82d5 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:22.127339 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd17c73-8962-4cab-88c8-5f159e8c82d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:22.146330 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-2bd17c73-8962-4cab-88c8-5f159e8c82d5 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:41:22.146593 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1086/2144] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:41:22 2022] POST /identity/v3/users => generated 290 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:41:22.147417 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64e97b6-d300-4d28-a892-835c8ceac0e2 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:22.147554 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-2bd17c73-8962-4cab-88c8-5f159e8c82d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:41:22.157567 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1059/2145] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:41:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:41:22.166790 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64e97b6-d300-4d28-a892-835c8ceac0e2 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:22.166934 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd17c73-8962-4cab-88c8-5f159e8c82d5 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:22.166934 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-2bd17c73-8962-4cab-88c8-5f159e8c82d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:41:22.167099 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8d6dd6-946b-4874-bdff-d8f65b9a6a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:22.167099 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d6dd6-946b-4874-bdff-d8f65b9a6a22 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:22.167099 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d6dd6-946b-4874-bdff-d8f65b9a6a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:22.167099 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d6dd6-946b-4874-bdff-d8f65b9a6a22 admin admin] PATH_INFO: `/v3/domains` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:22.167447 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8d6dd6-946b-4874-bdff-d8f65b9a6a22 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:41:22.167447 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8d6dd6-946b-4874-bdff-d8f65b9a6a22 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:22.168091 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8d6dd6-946b-4874-bdff-d8f65b9a6a22 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:22.172883 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd17c73-8962-4cab-88c8-5f159e8c82d5 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:22.177647 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f7d078-d6b7-4549-9b1e-088f9d1ae58d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:22.178360 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f7d078-d6b7-4549-9b1e-088f9d1ae58d admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:22.178651 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f7d078-d6b7-4549-9b1e-088f9d1ae58d admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:22.178935 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f7d078-d6b7-4549-9b1e-088f9d1ae58d admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:22.179677 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f7d078-d6b7-4549-9b1e-088f9d1ae58d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:22.181951 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f7d078-d6b7-4549-9b1e-088f9d1ae58d admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:22.182377 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1087/2146] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jun 7 14:41:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:41:22.183779 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3410c4-3112-42b9-82b0-5284bf766848 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:22.184105 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3410c4-3112-42b9-82b0-5284bf766848 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:22.184417 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3410c4-3112-42b9-82b0-5284bf766848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:22.196070 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1060/2147] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:41:22 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:41:22.197553 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f7d078-d6b7-4549-9b1e-088f9d1ae58d admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:22.204049 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-af3410c4-3112-42b9-82b0-5284bf766848 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:41:22.205318 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-af3410c4-3112-42b9-82b0-5284bf766848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:41:22.209107 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f7d078-d6b7-4549-9b1e-088f9d1ae58d admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:22.214064 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b572b0-43d3-4efb-9521-172a49c457eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:22.214803 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1088/2148] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:41:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:41:22.214947 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b572b0-43d3-4efb-9521-172a49c457eb admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:22.214947 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b572b0-43d3-4efb-9521-172a49c457eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:22.215224 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b572b0-43d3-4efb-9521-172a49c457eb admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:22.215560 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3410c4-3112-42b9-82b0-5284bf766848 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:22.216053 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b572b0-43d3-4efb-9521-172a49c457eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:22.218000 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b572b0-43d3-4efb-9521-172a49c457eb admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:22.226911 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3410c4-3112-42b9-82b0-5284bf766848 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:22.231879 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446aac81-1a06-482a-9c99-0d1bde570c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:22.232818 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446aac81-1a06-482a-9c99-0d1bde570c39 admin admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:22.233105 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446aac81-1a06-482a-9c99-0d1bde570c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:22.233385 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446aac81-1a06-482a-9c99-0d1bde570c39 admin admin] PATH_INFO: `/v3/projects/906c04b0c1834133bc19eab6f1f93f8f/users/3e76fa38c571489da81efc6c041f5b9c/roles/9baff2b18f2649f7b757ff2e1f63c853` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:22.248676 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446aac81-1a06-482a-9c99-0d1bde570c39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=906c04b0c1834133bc19eab6f1f93f8f, user_id=3e76fa38c571489da81efc6c041f5b9c, role_id=9baff2b18f2649f7b757ff2e1f63c853)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:22.251341 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446aac81-1a06-482a-9c99-0d1bde570c39 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:22.319702 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1061/2149] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:41:22 2022] POST /identity/v3/projects => generated 369 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:41:22.320941 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b572b0-43d3-4efb-9521-172a49c457eb admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:22.328790 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1089/2150] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:41:22 2022] PUT /identity/v3/projects/906c04b0c1834133bc19eab6f1f93f8f/users/3e76fa38c571489da81efc6c041f5b9c/roles/9baff2b18f2649f7b757ff2e1f63c853 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:41:22.329889 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446aac81-1a06-482a-9c99-0d1bde570c39 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:22.347823 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=94,keystone:DELETE=1,keystone:INSERT=5 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:41:22.400839 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:41:22.422905 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446aac81-1a06-482a-9c99-0d1bde570c39 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:22.430272 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b572b0-43d3-4efb-9521-172a49c457eb admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:22.432639 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ccd566-1014-4536-8eb6-85ab2ea12eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:22.433333 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ccd566-1014-4536-8eb6-85ab2ea12eaa admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:22.433622 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ccd566-1014-4536-8eb6-85ab2ea12eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:22.433908 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ccd566-1014-4536-8eb6-85ab2ea12eaa admin admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:22.434615 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ccd566-1014-4536-8eb6-85ab2ea12eaa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:22.436781 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ccd566-1014-4536-8eb6-85ab2ea12eaa admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:22.441522 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adab3386-de59-4669-9685-e54571e726da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:22.442214 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adab3386-de59-4669-9685-e54571e726da admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:22.442568 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adab3386-de59-4669-9685-e54571e726da admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:22.442961 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adab3386-de59-4669-9685-e54571e726da admin admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:22.443771 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adab3386-de59-4669-9685-e54571e726da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:41:22.444333 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adab3386-de59-4669-9685-e54571e726da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:22.446191 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adab3386-de59-4669-9685-e54571e726da admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:22.462096 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1062/2151] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jun 7 14:41:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:41:22.466327 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adab3386-de59-4669-9685-e54571e726da admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:22.476770 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adab3386-de59-4669-9685-e54571e726da admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:22.481619 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b4af2f-031e-4d4f-8caf-be40448ecb13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:22.482275 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b4af2f-031e-4d4f-8caf-be40448ecb13 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:22.482562 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b4af2f-031e-4d4f-8caf-be40448ecb13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:22.482845 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b4af2f-031e-4d4f-8caf-be40448ecb13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:22.483573 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b4af2f-031e-4d4f-8caf-be40448ecb13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:22.485197 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b4af2f-031e-4d4f-8caf-be40448ecb13 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:22.495607 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1063/2152] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:41:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:41:22.496454 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b4af2f-031e-4d4f-8caf-be40448ecb13 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:22.506365 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b4af2f-031e-4d4f-8caf-be40448ecb13 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:22.511115 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a099d1-0fde-46cc-bfc7-2e64c7f42143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:22.511763 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a099d1-0fde-46cc-bfc7-2e64c7f42143 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:22.512052 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a099d1-0fde-46cc-bfc7-2e64c7f42143 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:22.512334 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a099d1-0fde-46cc-bfc7-2e64c7f42143 admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:22.513056 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a099d1-0fde-46cc-bfc7-2e64c7f42143 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:22.515214 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a099d1-0fde-46cc-bfc7-2e64c7f42143 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:22.550916 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1090/2153] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:41:22 2022] POST /identity/v3/users => generated 285 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:41:22.552844 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ccd566-1014-4536-8eb6-85ab2ea12eaa admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:22.561548 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ccd566-1014-4536-8eb6-85ab2ea12eaa admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:22.566917 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eaeed3-f621-4d77-a4e1-aba30217f276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:22.567800 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eaeed3-f621-4d77-a4e1-aba30217f276 admin admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:22.568088 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eaeed3-f621-4d77-a4e1-aba30217f276 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:22.568393 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eaeed3-f621-4d77-a4e1-aba30217f276 admin admin] PATH_INFO: `/v3/projects/906c04b0c1834133bc19eab6f1f93f8f/users/3e76fa38c571489da81efc6c041f5b9c/roles/9baff2b18f2649f7b757ff2e1f63c853` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:22.570367 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eaeed3-f621-4d77-a4e1-aba30217f276 admin admin] RBAC: Authorizing `identity:create_grant(project_id=906c04b0c1834133bc19eab6f1f93f8f, user_id=3e76fa38c571489da81efc6c041f5b9c, role_id=9baff2b18f2649f7b757ff2e1f63c853)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:22.573074 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eaeed3-f621-4d77-a4e1-aba30217f276 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:22.584479 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1091/2154] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:41:22 2022] PUT /identity/v3/projects/906c04b0c1834133bc19eab6f1f93f8f/users/3e76fa38c571489da81efc6c041f5b9c/roles/9baff2b18f2649f7b757ff2e1f63c853 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:41:22.587116 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eaeed3-f621-4d77-a4e1-aba30217f276 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:22.612707 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1064/2155] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:41:22 2022] POST /identity/v3/projects => generated 375 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:41:22.612707 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c95e28-9836-43b9-837d-b7639530ba99 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:22.612707 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c95e28-9836-43b9-837d-b7639530ba99 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:22.612707 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c95e28-9836-43b9-837d-b7639530ba99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:22.640220 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-22c95e28-9836-43b9-837d-b7639530ba99 None None] MFA Rules not processed for user `3e76fa38c571489da81efc6c041f5b9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:41:22.641530 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-22c95e28-9836-43b9-837d-b7639530ba99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:41:22.654419 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eaeed3-f621-4d77-a4e1-aba30217f276 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:22.716084 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1065/2156] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:41:22 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:41:22.717117 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c95e28-9836-43b9-837d-b7639530ba99 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:22.724976 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56799cd-3b4b-45e5-b782-4c0d9349d451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:22.725885 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56799cd-3b4b-45e5-b782-4c0d9349d451 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:22.725993 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56799cd-3b4b-45e5-b782-4c0d9349d451 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:22.726220 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56799cd-3b4b-45e5-b782-4c0d9349d451 admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:22.729559 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56799cd-3b4b-45e5-b782-4c0d9349d451 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:22.731186 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56799cd-3b4b-45e5-b782-4c0d9349d451 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:22.750681 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1092/2157] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:41:22 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:41:22.750681 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56799cd-3b4b-45e5-b782-4c0d9349d451 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:22.751734 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-b56799cd-3b4b-45e5-b782-4c0d9349d451 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:41:22.792165 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:41:22.796715 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c95e28-9836-43b9-837d-b7639530ba99 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:22.801806 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac307c66-bf29-41e6-8f13-2b22d2c4be01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:22.802472 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac307c66-bf29-41e6-8f13-2b22d2c4be01 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:22.805197 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac307c66-bf29-41e6-8f13-2b22d2c4be01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:22.805197 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac307c66-bf29-41e6-8f13-2b22d2c4be01 admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:22.805197 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac307c66-bf29-41e6-8f13-2b22d2c4be01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:22.805916 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac307c66-bf29-41e6-8f13-2b22d2c4be01 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:22.818163 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56799cd-3b4b-45e5-b782-4c0d9349d451 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:22.822307 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25e56ac-3156-4c28-b36d-bfe9c200da52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d81f597cb44c56a966537089bcce4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:22.822962 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25e56ac-3156-4c28-b36d-bfe9c200da52 service cinder] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:22.823247 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25e56ac-3156-4c28-b36d-bfe9c200da52 service cinder] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:22.823624 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25e56ac-3156-4c28-b36d-bfe9c200da52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:22.834686 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25e56ac-3156-4c28-b36d-bfe9c200da52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:22.836218 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25e56ac-3156-4c28-b36d-bfe9c200da52 service cinder] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:22.846086 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1093/2158] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:41:22 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:41:22.847032 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25e56ac-3156-4c28-b36d-bfe9c200da52 service cinder] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:22.857366 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25e56ac-3156-4c28-b36d-bfe9c200da52 service cinder] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:22.862230 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e353ace9-761c-40ea-9598-b61fe85a34f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:22.862912 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e353ace9-761c-40ea-9598-b61fe85a34f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:22.863197 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e353ace9-761c-40ea-9598-b61fe85a34f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:22.863475 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e353ace9-761c-40ea-9598-b61fe85a34f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:22.864307 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e353ace9-761c-40ea-9598-b61fe85a34f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:22.865981 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e353ace9-761c-40ea-9598-b61fe85a34f5 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:22.876423 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1094/2159] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:41:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:41:22.881090 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e353ace9-761c-40ea-9598-b61fe85a34f5 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:22.891492 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e353ace9-761c-40ea-9598-b61fe85a34f5 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:22.897698 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a356209-26eb-4868-9b3b-550112778cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:22.898098 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a356209-26eb-4868-9b3b-550112778cf8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:22.898387 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a356209-26eb-4868-9b3b-550112778cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:22.898668 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a356209-26eb-4868-9b3b-550112778cf8 admin admin] PATH_INFO: `/v3/projects/6b5c0b32492048eab9662518d83ad6d6/users/a1a6625692d94e41890449d183379f38/roles/3771341b3be641139e752aab690397a4` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:22.914552 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a356209-26eb-4868-9b3b-550112778cf8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b5c0b32492048eab9662518d83ad6d6, user_id=a1a6625692d94e41890449d183379f38, role_id=3771341b3be641139e752aab690397a4)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:22.917283 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a356209-26eb-4868-9b3b-550112778cf8 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:22.926096 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1066/2160] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:41:22 2022] POST /identity/v3/users => generated 288 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:41:22.930283 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac307c66-bf29-41e6-8f13-2b22d2c4be01 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:22.941318 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac307c66-bf29-41e6-8f13-2b22d2c4be01 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:22.947478 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf93fb2-d259-492f-bea1-05963bb25948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:22.948298 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf93fb2-d259-492f-bea1-05963bb25948 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:22.948616 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf93fb2-d259-492f-bea1-05963bb25948 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:22.948970 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf93fb2-d259-492f-bea1-05963bb25948 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:22.949761 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf93fb2-d259-492f-bea1-05963bb25948 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:22.951868 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf93fb2-d259-492f-bea1-05963bb25948 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:22.974790 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1067/2161] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:41:22 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 14:41:22.975130 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf93fb2-d259-492f-bea1-05963bb25948 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:22.979701 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-eaf93fb2-d259-492f-bea1-05963bb25948 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:41:22.991018 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf93fb2-d259-492f-bea1-05963bb25948 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:22.995340 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bba06f-4c8d-4f80-bc02-bc2c090863b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d81f597cb44c56a966537089bcce4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:22.995995 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bba06f-4c8d-4f80-bc02-bc2c090863b6 service cinder] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:22.996279 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bba06f-4c8d-4f80-bc02-bc2c090863b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:22.996594 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bba06f-4c8d-4f80-bc02-bc2c090863b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:22.998383 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-57bba06f-4c8d-4f80-bc02-bc2c090863b6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:41:23.008514 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1095/2162] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:41:22 2022] PUT /identity/v3/projects/6b5c0b32492048eab9662518d83ad6d6/users/a1a6625692d94e41890449d183379f38/roles/3771341b3be641139e752aab690397a4 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:41:23.008910 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:41:23.011800 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a356209-26eb-4868-9b3b-550112778cf8 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:23.076221 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bba06f-4c8d-4f80-bc02-bc2c090863b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:23.077895 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bba06f-4c8d-4f80-bc02-bc2c090863b6 service cinder] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:23.088504 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1068/2163] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:41:22 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:41:23.089814 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e67a5d-4d2e-4fa1-90dd-d5652f112ddb None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:23.090108 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e67a5d-4d2e-4fa1-90dd-d5652f112ddb None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:23.090389 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e67a5d-4d2e-4fa1-90dd-d5652f112ddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:23.111642 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a356209-26eb-4868-9b3b-550112778cf8 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:23.116608 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801847a1-4fa8-4183-8d13-a86356a2e9aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:23.117474 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801847a1-4fa8-4183-8d13-a86356a2e9aa admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:23.117754 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801847a1-4fa8-4183-8d13-a86356a2e9aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:23.118178 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801847a1-4fa8-4183-8d13-a86356a2e9aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:23.118805 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-d9e67a5d-4d2e-4fa1-90dd-d5652f112ddb None None] MFA Rules not processed for user `a1a6625692d94e41890449d183379f38`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:41:23.118933 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801847a1-4fa8-4183-8d13-a86356a2e9aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:23.120017 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-d9e67a5d-4d2e-4fa1-90dd-d5652f112ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:41:23.120557 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801847a1-4fa8-4183-8d13-a86356a2e9aa admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:23.132665 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1096/2164] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:41:23 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:41:23.142494 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801847a1-4fa8-4183-8d13-a86356a2e9aa admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:23.153370 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801847a1-4fa8-4183-8d13-a86356a2e9aa admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:23.158357 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd52887-a51d-4010-a0d4-d1f80c32c553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:23.160029 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd52887-a51d-4010-a0d4-d1f80c32c553 admin admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:23.160245 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd52887-a51d-4010-a0d4-d1f80c32c553 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:23.160344 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd52887-a51d-4010-a0d4-d1f80c32c553 admin admin] PATH_INFO: `/v3/projects/39ef8e106ba4454aab0ae9410aa51f0d/users/c813f52639e24cec86b64eeae4d99b60/roles/3771341b3be641139e752aab690397a4` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:23.183355 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd52887-a51d-4010-a0d4-d1f80c32c553 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39ef8e106ba4454aab0ae9410aa51f0d, user_id=c813f52639e24cec86b64eeae4d99b60, role_id=3771341b3be641139e752aab690397a4)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:23.186241 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd52887-a51d-4010-a0d4-d1f80c32c553 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:23.213491 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1069/2165] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:41:23 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:41:23.225326 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e67a5d-4d2e-4fa1-90dd-d5652f112ddb None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:23.228202 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-d9e67a5d-4d2e-4fa1-90dd-d5652f112ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:41:23.290613 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1097/2166] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:41:23 2022] PUT /identity/v3/projects/39ef8e106ba4454aab0ae9410aa51f0d/users/c813f52639e24cec86b64eeae4d99b60/roles/3771341b3be641139e752aab690397a4 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:41:23.294593 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e67a5d-4d2e-4fa1-90dd-d5652f112ddb None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:23.302014 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad83ccf-e1f9-4b0c-8b2b-8ba5a20114c1 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:23.302327 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad83ccf-e1f9-4b0c-8b2b-8ba5a20114c1 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:23.302573 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad83ccf-e1f9-4b0c-8b2b-8ba5a20114c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:23.336027 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-2ad83ccf-e1f9-4b0c-8b2b-8ba5a20114c1 None None] MFA Rules not processed for user `c813f52639e24cec86b64eeae4d99b60`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:41:23.337416 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-2ad83ccf-e1f9-4b0c-8b2b-8ba5a20114c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:41:23.371224 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a97b9a0-db11-4a52-a894-05e53f8ca475 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51da9867d9147f198b8077e953c4f9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:23.372454 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a97b9a0-db11-4a52-a894-05e53f8ca475 service glance] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:23.372962 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a97b9a0-db11-4a52-a894-05e53f8ca475 service glance] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:23.373433 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a97b9a0-db11-4a52-a894-05e53f8ca475 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:23.386726 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:41:23.433829 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:41:23.448413 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1098/2167] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:41:23 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:41:23.454339 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad83ccf-e1f9-4b0c-8b2b-8ba5a20114c1 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:23.501988 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a97b9a0-db11-4a52-a894-05e53f8ca475 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:23.505935 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a97b9a0-db11-4a52-a894-05e53f8ca475 service glance] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:23.529798 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1070/2168] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:41:23 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 14:41:23.551265 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad83ccf-e1f9-4b0c-8b2b-8ba5a20114c1 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:23.559275 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379fbe17-0344-4dcb-ba33-aa45b6f5de32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:23.560464 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379fbe17-0344-4dcb-ba33-aa45b6f5de32 admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:23.560990 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379fbe17-0344-4dcb-ba33-aa45b6f5de32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:23.561483 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379fbe17-0344-4dcb-ba33-aa45b6f5de32 admin admin] PATH_INFO: `/v3/projects` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:23.563360 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379fbe17-0344-4dcb-ba33-aa45b6f5de32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:23.566265 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379fbe17-0344-4dcb-ba33-aa45b6f5de32 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:23.687617 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1099/2169] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:41:23 2022] POST /identity/v3/projects => generated 375 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:41:23.691500 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a97b9a0-db11-4a52-a894-05e53f8ca475 service glance] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:23.722063 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a97b9a0-db11-4a52-a894-05e53f8ca475 service glance] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:23.819670 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84f8798-23ba-4c1c-8bc6-97473719b0d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:23.819670 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84f8798-23ba-4c1c-8bc6-97473719b0d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:23.819670 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84f8798-23ba-4c1c-8bc6-97473719b0d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:23.819670 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84f8798-23ba-4c1c-8bc6-97473719b0d1 admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:23.830891 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84f8798-23ba-4c1c-8bc6-97473719b0d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:23.830891 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84f8798-23ba-4c1c-8bc6-97473719b0d1 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:23.951541 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1071/2170] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:41:23 2022] POST /identity/v3/users => generated 288 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:41:23.963314 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379fbe17-0344-4dcb-ba33-aa45b6f5de32 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:23.983917 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379fbe17-0344-4dcb-ba33-aa45b6f5de32 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:23.996581 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b41f95-e814-4461-9aba-7c2130017195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:23.997853 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b41f95-e814-4461-9aba-7c2130017195 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:23.998344 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b41f95-e814-4461-9aba-7c2130017195 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:23.998824 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b41f95-e814-4461-9aba-7c2130017195 admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:24.000134 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b41f95-e814-4461-9aba-7c2130017195 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:24.007673 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b41f95-e814-4461-9aba-7c2130017195 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:24.024345 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1100/2171] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:41:23 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:41:24.028019 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84f8798-23ba-4c1c-8bc6-97473719b0d1 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:24.042275 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84f8798-23ba-4c1c-8bc6-97473719b0d1 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:24.049250 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e34790-fdc9-437e-9ddb-a66fc2f2bdea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:24.050476 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e34790-fdc9-437e-9ddb-a66fc2f2bdea admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:24.050764 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e34790-fdc9-437e-9ddb-a66fc2f2bdea admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:24.051043 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e34790-fdc9-437e-9ddb-a66fc2f2bdea admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:24.051783 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e34790-fdc9-437e-9ddb-a66fc2f2bdea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:24.055443 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e34790-fdc9-437e-9ddb-a66fc2f2bdea admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:24.071977 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1072/2172] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:41:24 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:41:24.073265 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b41f95-e814-4461-9aba-7c2130017195 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:24.084797 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b41f95-e814-4461-9aba-7c2130017195 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:24.093962 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266216e3-4310-4469-ba5f-5faff5544e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:24.094997 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266216e3-4310-4469-ba5f-5faff5544e2d admin admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:24.102015 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266216e3-4310-4469-ba5f-5faff5544e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:24.102015 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266216e3-4310-4469-ba5f-5faff5544e2d admin admin] PATH_INFO: `/v3/projects/03026ac1e53f4518b19f91bf7fa954cc/users/db5101ff94444707a2fa0548bd92b818/roles/3771341b3be641139e752aab690397a4` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:24.133514 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266216e3-4310-4469-ba5f-5faff5544e2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=03026ac1e53f4518b19f91bf7fa954cc, user_id=db5101ff94444707a2fa0548bd92b818, role_id=3771341b3be641139e752aab690397a4)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:24.138825 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266216e3-4310-4469-ba5f-5faff5544e2d admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:24.211413 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1101/2173] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:41:24 2022] PUT /identity/v3/projects/03026ac1e53f4518b19f91bf7fa954cc/users/db5101ff94444707a2fa0548bd92b818/roles/3771341b3be641139e752aab690397a4 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:41:24.224637 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538f7f5b-0b94-4176-8b54-65cfbbe91dda None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:24.224958 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538f7f5b-0b94-4176-8b54-65cfbbe91dda None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:24.225733 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538f7f5b-0b94-4176-8b54-65cfbbe91dda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:24.258683 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-538f7f5b-0b94-4176-8b54-65cfbbe91dda None None] MFA Rules not processed for user `db5101ff94444707a2fa0548bd92b818`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:41:24.259939 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-538f7f5b-0b94-4176-8b54-65cfbbe91dda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:41:24.374043 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1073/2174] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:41:24 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:41:24.379485 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266216e3-4310-4469-ba5f-5faff5544e2d admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:24.481236 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266216e3-4310-4469-ba5f-5faff5544e2d admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:24.486748 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd682e85-1e99-4301-8d25-dba53048f288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:24.488188 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd682e85-1e99-4301-8d25-dba53048f288 admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:24.488621 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd682e85-1e99-4301-8d25-dba53048f288 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:24.488969 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd682e85-1e99-4301-8d25-dba53048f288 admin admin] PATH_INFO: `/v3/projects` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:24.489806 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd682e85-1e99-4301-8d25-dba53048f288 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:24.491943 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd682e85-1e99-4301-8d25-dba53048f288 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:24.596042 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1102/2175] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:41:24 2022] POST /identity/v3/projects => generated 377 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:41:24.599896 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538f7f5b-0b94-4176-8b54-65cfbbe91dda None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:24.614304 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538f7f5b-0b94-4176-8b54-65cfbbe91dda None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:24.630468 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:41:24.700417 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e8fb36-f4c2-4d7a-90f5-2f26c3726e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:24.701114 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e8fb36-f4c2-4d7a-90f5-2f26c3726e77 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:24.701400 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e8fb36-f4c2-4d7a-90f5-2f26c3726e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:24.701679 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e8fb36-f4c2-4d7a-90f5-2f26c3726e77 admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:24.702373 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e8fb36-f4c2-4d7a-90f5-2f26c3726e77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:24.704336 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e8fb36-f4c2-4d7a-90f5-2f26c3726e77 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:24.791884 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1074/2176] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:41:24 2022] POST /identity/v3/users => generated 295 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:41:24.795909 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd682e85-1e99-4301-8d25-dba53048f288 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:24.808760 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd682e85-1e99-4301-8d25-dba53048f288 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:24.813790 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b52d72-bf0d-4a87-bb7f-af4e24c1c64f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:24.814570 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b52d72-bf0d-4a87-bb7f-af4e24c1c64f admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:24.814864 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b52d72-bf0d-4a87-bb7f-af4e24c1c64f admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:24.815147 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b52d72-bf0d-4a87-bb7f-af4e24c1c64f admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:24.816527 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b52d72-bf0d-4a87-bb7f-af4e24c1c64f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:24.818250 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b52d72-bf0d-4a87-bb7f-af4e24c1c64f admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:24.829661 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1103/2177] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:41:24 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 14:41:24.833920 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e8fb36-f4c2-4d7a-90f5-2f26c3726e77 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:24.844450 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e8fb36-f4c2-4d7a-90f5-2f26c3726e77 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:24.850073 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e557e2-9329-470c-899f-a6603d7daad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:24.850985 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e557e2-9329-470c-899f-a6603d7daad8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:24.851274 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e557e2-9329-470c-899f-a6603d7daad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:24.851551 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e557e2-9329-470c-899f-a6603d7daad8 admin admin] PATH_INFO: `/v3/projects/e3d47631ae6f46f3993da856993fe000/users/481331f1597e4076bc16bbe05e5060d5/roles/9baff2b18f2649f7b757ff2e1f63c853` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:24.871802 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e557e2-9329-470c-899f-a6603d7daad8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3d47631ae6f46f3993da856993fe000, user_id=481331f1597e4076bc16bbe05e5060d5, role_id=9baff2b18f2649f7b757ff2e1f63c853)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:24.875426 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e557e2-9329-470c-899f-a6603d7daad8 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:24.967704 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1075/2178] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:41:24 2022] PUT /identity/v3/projects/e3d47631ae6f46f3993da856993fe000/users/481331f1597e4076bc16bbe05e5060d5/roles/9baff2b18f2649f7b757ff2e1f63c853 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:41:24.970511 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b52d72-bf0d-4a87-bb7f-af4e24c1c64f admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:24.981583 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b52d72-bf0d-4a87-bb7f-af4e24c1c64f admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:25.003032 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:41:25.046880 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc2216d-5f28-498b-abde-76beb083f62e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:25.047571 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc2216d-5f28-498b-abde-76beb083f62e admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:25.047911 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc2216d-5f28-498b-abde-76beb083f62e admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:25.048220 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc2216d-5f28-498b-abde-76beb083f62e admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:25.048965 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc2216d-5f28-498b-abde-76beb083f62e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:25.050812 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc2216d-5f28-498b-abde-76beb083f62e admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:25.067887 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1104/2179] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:41:24 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:41:25.069443 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e557e2-9329-470c-899f-a6603d7daad8 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:25.083457 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e557e2-9329-470c-899f-a6603d7daad8 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:25.106761 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b43f467-3193-4698-8148-917cabb37925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:25.107686 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b43f467-3193-4698-8148-917cabb37925 admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:25.107970 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b43f467-3193-4698-8148-917cabb37925 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:25.108249 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b43f467-3193-4698-8148-917cabb37925 admin admin] PATH_INFO: `/v3/projects/e3d47631ae6f46f3993da856993fe000/users/481331f1597e4076bc16bbe05e5060d5/roles/9baff2b18f2649f7b757ff2e1f63c853` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:25.110625 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b43f467-3193-4698-8148-917cabb37925 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3d47631ae6f46f3993da856993fe000, user_id=481331f1597e4076bc16bbe05e5060d5, role_id=9baff2b18f2649f7b757ff2e1f63c853)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:25.116135 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b43f467-3193-4698-8148-917cabb37925 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:25.133240 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1076/2180] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:41:25 2022] PUT /identity/v3/projects/e3d47631ae6f46f3993da856993fe000/users/481331f1597e4076bc16bbe05e5060d5/roles/9baff2b18f2649f7b757ff2e1f63c853 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:41:25.143376 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869ea6c4-cc03-41a5-bef9-87105832671e None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:25.148795 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869ea6c4-cc03-41a5-bef9-87105832671e None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:25.148795 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869ea6c4-cc03-41a5-bef9-87105832671e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:25.188198 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-869ea6c4-cc03-41a5-bef9-87105832671e None None] MFA Rules not processed for user `481331f1597e4076bc16bbe05e5060d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:41:25.189719 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-869ea6c4-cc03-41a5-bef9-87105832671e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:41:25.297339 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1105/2181] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:41:25 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:41:25.310867 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b43f467-3193-4698-8148-917cabb37925 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:25.378852 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b43f467-3193-4698-8148-917cabb37925 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:25.384779 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0866b636-39ca-44fa-8858-653d7d6753fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d81f597cb44c56a966537089bcce4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:25.385817 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0866b636-39ca-44fa-8858-653d7d6753fb service cinder] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:25.386289 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0866b636-39ca-44fa-8858-653d7d6753fb service cinder] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:25.386773 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0866b636-39ca-44fa-8858-653d7d6753fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:25.478847 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0866b636-39ca-44fa-8858-653d7d6753fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:25.480306 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0866b636-39ca-44fa-8858-653d7d6753fb service cinder] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:25.491622 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1077/2182] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:41:25 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:41:28.425943 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869ea6c4-cc03-41a5-bef9-87105832671e None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:28.497207 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869ea6c4-cc03-41a5-bef9-87105832671e None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:28.501727 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5e7263-0514-4dc3-b79b-809eb1b73951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:28.502609 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5e7263-0514-4dc3-b79b-809eb1b73951 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:28.502896 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5e7263-0514-4dc3-b79b-809eb1b73951 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:28.503737 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5e7263-0514-4dc3-b79b-809eb1b73951 admin admin] PATH_INFO: `/v3/users/4c2185768fe04873a227855201b7234d` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:28.505028 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5e7263-0514-4dc3-b79b-809eb1b73951 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c2185768fe04873a227855201b7234d)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:28.507017 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5e7263-0514-4dc3-b79b-809eb1b73951 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:28.628734 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0866b636-39ca-44fa-8858-653d7d6753fb service cinder] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:28.638831 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0866b636-39ca-44fa-8858-653d7d6753fb service cinder] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:28.643267 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa2c77d-6c8b-4bb6-99b7-48d40c055540 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d81f597cb44c56a966537089bcce4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:28.644555 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa2c77d-6c8b-4bb6-99b7-48d40c055540 service cinder] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:28.644840 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa2c77d-6c8b-4bb6-99b7-48d40c055540 service cinder] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:28.645893 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa2c77d-6c8b-4bb6-99b7-48d40c055540 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:28.667554 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:41:28.705557 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa2c77d-6c8b-4bb6-99b7-48d40c055540 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:28.707579 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa2c77d-6c8b-4bb6-99b7-48d40c055540 service cinder] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:28.718521 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1078/2183] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:41:28 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 14:41:28.743753 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-2c5e7263-0514-4dc3-b79b-809eb1b73951 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c2185768fe04873a227855201b7234d'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:28.744562 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-2c5e7263-0514-4dc3-b79b-809eb1b73951 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c2185768fe04873a227855201b7234d'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:28.745259 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-2c5e7263-0514-4dc3-b79b-809eb1b73951 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c2185768fe04873a227855201b7234d'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:28.746627 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:41:28.752874 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-2c5e7263-0514-4dc3-b79b-809eb1b73951 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c2185768fe04873a227855201b7234d'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:28.765783 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-2c5e7263-0514-4dc3-b79b-809eb1b73951 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c2185768fe04873a227855201b7234d'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:28.855184 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1106/2184] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:41:28 2022] DELETE /identity/v3/users/4c2185768fe04873a227855201b7234d => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:41:28.860616 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea9ed00-0362-48f3-ba44-4660c16203c3 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:28.860616 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea9ed00-0362-48f3-ba44-4660c16203c3 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:28.860616 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea9ed00-0362-48f3-ba44-4660c16203c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:28.881019 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-9ea9ed00-0362-48f3-ba44-4660c16203c3 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:41:28.882229 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-9ea9ed00-0362-48f3-ba44-4660c16203c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:41:28.951142 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1079/2185] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:41:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:41:28.966286 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5e7263-0514-4dc3-b79b-809eb1b73951 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:28.967641 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-2c5e7263-0514-4dc3-b79b-809eb1b73951 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:41:29.031094 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5e7263-0514-4dc3-b79b-809eb1b73951 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:29.036626 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acbf6fd-5d81-44c6-898d-4702444f8ecd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:29.043565 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acbf6fd-5d81-44c6-898d-4702444f8ecd service neutron] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:29.043565 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acbf6fd-5d81-44c6-898d-4702444f8ecd service neutron] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:29.043565 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acbf6fd-5d81-44c6-898d-4702444f8ecd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:29.047842 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acbf6fd-5d81-44c6-898d-4702444f8ecd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:29.049416 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acbf6fd-5d81-44c6-898d-4702444f8ecd service neutron] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:29.060463 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1107/2186] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:41:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:41:29.543210 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea9ed00-0362-48f3-ba44-4660c16203c3 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:29.545069 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-9ea9ed00-0362-48f3-ba44-4660c16203c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:41:29.563498 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea9ed00-0362-48f3-ba44-4660c16203c3 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:29.568541 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a07f923-c50e-446b-99c8-5793d28c8da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:29.569483 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a07f923-c50e-446b-99c8-5793d28c8da2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:29.569770 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a07f923-c50e-446b-99c8-5793d28c8da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:29.577428 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a07f923-c50e-446b-99c8-5793d28c8da2 admin admin] PATH_INFO: `/v3/projects/a95d925b241c477db7f0e8ca7fdef3ee` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:29.577428 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a07f923-c50e-446b-99c8-5793d28c8da2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a95d925b241c477db7f0e8ca7fdef3ee)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:29.577428 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a07f923-c50e-446b-99c8-5793d28c8da2 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:29.716185 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-8a07f923-c50e-446b-99c8-5793d28c8da2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a95d925b241c477db7f0e8ca7fdef3ee', 'request_id': 'req-8a07f923-c50e-446b-99c8-5793d28c8da2'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:29.801029 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-8a07f923-c50e-446b-99c8-5793d28c8da2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a95d925b241c477db7f0e8ca7fdef3ee', 'request_id': 'req-8a07f923-c50e-446b-99c8-5793d28c8da2'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:29.807724 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-8a07f923-c50e-446b-99c8-5793d28c8da2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a95d925b241c477db7f0e8ca7fdef3ee', 'request_id': 'req-8a07f923-c50e-446b-99c8-5793d28c8da2'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:29.819923 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-8a07f923-c50e-446b-99c8-5793d28c8da2 admin admin] The token cache is being invalidate because project a95d925b241c477db7f0e8ca7fdef3ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:41:29.820214 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-8a07f923-c50e-446b-99c8-5793d28c8da2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:29.821007 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-8a07f923-c50e-446b-99c8-5793d28c8da2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:29.823325 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1080/2187] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:41:29 2022] DELETE /identity/v3/projects/a95d925b241c477db7f0e8ca7fdef3ee => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:41:29.827393 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acbf6fd-5d81-44c6-898d-4702444f8ecd service neutron] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:29.838346 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acbf6fd-5d81-44c6-898d-4702444f8ecd service neutron] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:29.908162 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6adb85a-75a9-495a-b9b7-781d3be407be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:29.909087 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6adb85a-75a9-495a-b9b7-781d3be407be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:29.909921 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6adb85a-75a9-495a-b9b7-781d3be407be admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:29.910264 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6adb85a-75a9-495a-b9b7-781d3be407be admin admin] PATH_INFO: `/v3/users/3e76fa38c571489da81efc6c041f5b9c` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:29.911561 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6adb85a-75a9-495a-b9b7-781d3be407be admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e76fa38c571489da81efc6c041f5b9c)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:29.913530 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6adb85a-75a9-495a-b9b7-781d3be407be admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:30.082710 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-c6adb85a-75a9-495a-b9b7-781d3be407be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e76fa38c571489da81efc6c041f5b9c'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:30.083673 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-c6adb85a-75a9-495a-b9b7-781d3be407be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e76fa38c571489da81efc6c041f5b9c'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:30.084301 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-c6adb85a-75a9-495a-b9b7-781d3be407be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e76fa38c571489da81efc6c041f5b9c'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:30.092504 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-c6adb85a-75a9-495a-b9b7-781d3be407be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e76fa38c571489da81efc6c041f5b9c'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:30.103634 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:41:30.106224 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-c6adb85a-75a9-495a-b9b7-781d3be407be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e76fa38c571489da81efc6c041f5b9c'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:30.175439 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1108/2188] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:41:29 2022] DELETE /identity/v3/users/3e76fa38c571489da81efc6c041f5b9c => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:41:30.454628 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a07f923-c50e-446b-99c8-5793d28c8da2 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:30.458760 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-8a07f923-c50e-446b-99c8-5793d28c8da2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:41:30.474511 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a07f923-c50e-446b-99c8-5793d28c8da2 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:30.491213 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6adb85a-75a9-495a-b9b7-781d3be407be admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:30.493328 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-c6adb85a-75a9-495a-b9b7-781d3be407be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:41:30.555078 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:41:30.579763 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e722ef-e21a-4625-9e42-0418b4206a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:30.581486 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e722ef-e21a-4625-9e42-0418b4206a66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:30.581994 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e722ef-e21a-4625-9e42-0418b4206a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:30.582362 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e722ef-e21a-4625-9e42-0418b4206a66 admin admin] PATH_INFO: `/v3/users/970b2ae5af3c499495085e85a51f3c05` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:30.584899 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e722ef-e21a-4625-9e42-0418b4206a66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=970b2ae5af3c499495085e85a51f3c05)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:30.593657 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e722ef-e21a-4625-9e42-0418b4206a66 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:30.606714 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6adb85a-75a9-495a-b9b7-781d3be407be admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:30.611335 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edd47f6-7e03-40ee-89d9-b111d6dbaa0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d81f597cb44c56a966537089bcce4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:30.612106 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd47f6-7e03-40ee-89d9-b111d6dbaa0c service cinder] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:30.612999 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd47f6-7e03-40ee-89d9-b111d6dbaa0c service cinder] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:30.613451 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd47f6-7e03-40ee-89d9-b111d6dbaa0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:30.616992 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-8edd47f6-7e03-40ee-89d9-b111d6dbaa0c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:41:30.688231 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edd47f6-7e03-40ee-89d9-b111d6dbaa0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:30.690485 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edd47f6-7e03-40ee-89d9-b111d6dbaa0c service cinder] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:30.700995 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1109/2189] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:41:30 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 14:41:30.722868 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edd47f6-7e03-40ee-89d9-b111d6dbaa0c service cinder] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:30.727993 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-8edd47f6-7e03-40ee-89d9-b111d6dbaa0c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:41:30.746048 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edd47f6-7e03-40ee-89d9-b111d6dbaa0c service cinder] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:30.750977 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f6d6b9-49b7-42ef-952a-3ceb5435c970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:30.751916 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f6d6b9-49b7-42ef-952a-3ceb5435c970 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:30.753028 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f6d6b9-49b7-42ef-952a-3ceb5435c970 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:30.753028 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f6d6b9-49b7-42ef-952a-3ceb5435c970 admin admin] PATH_INFO: `/v3/projects/906c04b0c1834133bc19eab6f1f93f8f` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:30.753948 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f6d6b9-49b7-42ef-952a-3ceb5435c970 admin admin] RBAC: Authorizing `identity:delete_project(project_id=906c04b0c1834133bc19eab6f1f93f8f)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:30.756105 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f6d6b9-49b7-42ef-952a-3ceb5435c970 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:30.850195 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-b6e722ef-e21a-4625-9e42-0418b4206a66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '970b2ae5af3c499495085e85a51f3c05'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:30.851096 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-b6e722ef-e21a-4625-9e42-0418b4206a66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '970b2ae5af3c499495085e85a51f3c05'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:30.851723 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-b6e722ef-e21a-4625-9e42-0418b4206a66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '970b2ae5af3c499495085e85a51f3c05'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:30.861058 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-b6e722ef-e21a-4625-9e42-0418b4206a66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '970b2ae5af3c499495085e85a51f3c05'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:30.873284 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-b6e722ef-e21a-4625-9e42-0418b4206a66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '970b2ae5af3c499495085e85a51f3c05'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:30.983313 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1081/2190] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:41:30 2022] DELETE /identity/v3/users/970b2ae5af3c499495085e85a51f3c05 => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:41:30.987537 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325a6af8-573b-4f2a-b56f-f5e4737bcaf0 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:30.987927 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325a6af8-573b-4f2a-b56f-f5e4737bcaf0 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:30.988805 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325a6af8-573b-4f2a-b56f-f5e4737bcaf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:30.988948 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-10f6d6b9-49b7-42ef-952a-3ceb5435c970 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '906c04b0c1834133bc19eab6f1f93f8f', 'request_id': 'req-10f6d6b9-49b7-42ef-952a-3ceb5435c970'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:31.013363 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-325a6af8-573b-4f2a-b56f-f5e4737bcaf0 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:41:31.014197 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-325a6af8-573b-4f2a-b56f-f5e4737bcaf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:41:31.084887 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1082/2191] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:41:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:41:31.095137 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325a6af8-573b-4f2a-b56f-f5e4737bcaf0 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:31.096971 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-325a6af8-573b-4f2a-b56f-f5e4737bcaf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:41:31.100291 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-10f6d6b9-49b7-42ef-952a-3ceb5435c970 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '906c04b0c1834133bc19eab6f1f93f8f', 'request_id': 'req-10f6d6b9-49b7-42ef-952a-3ceb5435c970'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:31.109908 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-10f6d6b9-49b7-42ef-952a-3ceb5435c970 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '906c04b0c1834133bc19eab6f1f93f8f', 'request_id': 'req-10f6d6b9-49b7-42ef-952a-3ceb5435c970'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:31.121753 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-10f6d6b9-49b7-42ef-952a-3ceb5435c970 admin admin] The token cache is being invalidate because project 906c04b0c1834133bc19eab6f1f93f8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:41:31.122063 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-10f6d6b9-49b7-42ef-952a-3ceb5435c970 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:31.122747 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-10f6d6b9-49b7-42ef-952a-3ceb5435c970 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:31.124339 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1110/2192] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:41:30 2022] DELETE /identity/v3/projects/906c04b0c1834133bc19eab6f1f93f8f => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:41:31.129472 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e763ce-8c55-469e-b431-97b91f9e92fb None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:31.129766 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e763ce-8c55-469e-b431-97b91f9e92fb None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:31.130052 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e763ce-8c55-469e-b431-97b91f9e92fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:31.162744 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-f9e763ce-8c55-469e-b431-97b91f9e92fb None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:41:31.165762 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-f9e763ce-8c55-469e-b431-97b91f9e92fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:41:31.181214 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1111/2193] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:41:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:41:31.184170 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5949ccf-f904-451c-8856-837ac6024335 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:31.184542 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5949ccf-f904-451c-8856-837ac6024335 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:31.184835 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5949ccf-f904-451c-8856-837ac6024335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:31.188640 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325a6af8-573b-4f2a-b56f-f5e4737bcaf0 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:31.193061 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:41:31.195869 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272a002c-4690-445b-851c-3a69bd4b949b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:31.196822 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272a002c-4690-445b-851c-3a69bd4b949b service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:31.197339 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272a002c-4690-445b-851c-3a69bd4b949b service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:31.197828 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272a002c-4690-445b-851c-3a69bd4b949b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:31.201847 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-272a002c-4690-445b-851c-3a69bd4b949b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:41:31.218141 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-d5949ccf-f904-451c-8856-837ac6024335 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:41:31.218141 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-d5949ccf-f904-451c-8856-837ac6024335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:41:31.218703 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272a002c-4690-445b-851c-3a69bd4b949b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:31.220697 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272a002c-4690-445b-851c-3a69bd4b949b service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:31.236641 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1112/2194] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:41:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:41:31.238615 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5949ccf-f904-451c-8856-837ac6024335 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:31.251358 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5949ccf-f904-451c-8856-837ac6024335 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:31.258036 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beed84dc-2a3a-48ea-8ec1-0d9fa6078cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:31.259133 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beed84dc-2a3a-48ea-8ec1-0d9fa6078cab admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:31.259693 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beed84dc-2a3a-48ea-8ec1-0d9fa6078cab admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:31.260129 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beed84dc-2a3a-48ea-8ec1-0d9fa6078cab admin admin] PATH_INFO: `/v3/domains` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:31.260630 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1083/2195] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:41:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 14:41:31.261490 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beed84dc-2a3a-48ea-8ec1-0d9fa6078cab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:41:31.261876 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beed84dc-2a3a-48ea-8ec1-0d9fa6078cab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:31.263587 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beed84dc-2a3a-48ea-8ec1-0d9fa6078cab admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:31.281584 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1113/2196] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jun 7 14:41:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:41:31.286112 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272a002c-4690-445b-851c-3a69bd4b949b service neutron] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:31.298045 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272a002c-4690-445b-851c-3a69bd4b949b service neutron] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:31.305493 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c02c51-94e4-48c5-838b-da754535e4b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:31.310209 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c02c51-94e4-48c5-838b-da754535e4b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:31.310209 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c02c51-94e4-48c5-838b-da754535e4b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:31.310209 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c02c51-94e4-48c5-838b-da754535e4b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:31.310209 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c02c51-94e4-48c5-838b-da754535e4b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:31.311920 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c02c51-94e4-48c5-838b-da754535e4b5 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:31.317923 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:41:31.442683 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1084/2197] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:41:31 2022] POST /identity/v3/projects => generated 393 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:41:31.444200 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beed84dc-2a3a-48ea-8ec1-0d9fa6078cab admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:31.454410 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beed84dc-2a3a-48ea-8ec1-0d9fa6078cab admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:31.530325 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be375a57-b325-48dc-9b94-0b4c964ab42a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:31.531119 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be375a57-b325-48dc-9b94-0b4c964ab42a admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:31.531503 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be375a57-b325-48dc-9b94-0b4c964ab42a admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:31.531974 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be375a57-b325-48dc-9b94-0b4c964ab42a admin admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:31.533173 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be375a57-b325-48dc-9b94-0b4c964ab42a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:31.536260 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be375a57-b325-48dc-9b94-0b4c964ab42a admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:31.654280 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1114/2198] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:41:31 2022] POST /identity/v3/users => generated 297 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:41:31.657664 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c02c51-94e4-48c5-838b-da754535e4b5 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:31.672220 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c02c51-94e4-48c5-838b-da754535e4b5 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:31.680603 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd121c8-18f3-479c-8951-aba1b7027a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:31.683144 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd121c8-18f3-479c-8951-aba1b7027a41 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:31.683144 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd121c8-18f3-479c-8951-aba1b7027a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:31.683144 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd121c8-18f3-479c-8951-aba1b7027a41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:31.684060 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd121c8-18f3-479c-8951-aba1b7027a41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:31.688630 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd121c8-18f3-479c-8951-aba1b7027a41 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:31.728476 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1085/2199] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:41:31 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:41:31.729248 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be375a57-b325-48dc-9b94-0b4c964ab42a admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:31.740808 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be375a57-b325-48dc-9b94-0b4c964ab42a admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:31.746536 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b96501d-a499-4b32-8697-c3dee215531b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:31.747217 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b96501d-a499-4b32-8697-c3dee215531b admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:31.747513 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b96501d-a499-4b32-8697-c3dee215531b admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:31.747798 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b96501d-a499-4b32-8697-c3dee215531b admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:31.748753 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b96501d-a499-4b32-8697-c3dee215531b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:31.750382 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b96501d-a499-4b32-8697-c3dee215531b admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:31.761806 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd121c8-18f3-479c-8951-aba1b7027a41 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:31.763404 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-3dd121c8-18f3-479c-8951-aba1b7027a41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:41:31.779243 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1115/2200] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:41:31 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 14:41:31.795270 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd121c8-18f3-479c-8951-aba1b7027a41 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:31.800350 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b96501d-a499-4b32-8697-c3dee215531b admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:31.801052 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608c7112-c23a-47cd-a46a-1504886ba13e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:31.802026 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c7112-c23a-47cd-a46a-1504886ba13e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:31.802277 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c7112-c23a-47cd-a46a-1504886ba13e admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:31.802619 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c7112-c23a-47cd-a46a-1504886ba13e admin admin] PATH_INFO: `/v3/projects/19128ae8ce544f70a0fd148c537ca15b` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:31.804779 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608c7112-c23a-47cd-a46a-1504886ba13e admin admin] RBAC: Authorizing `identity:delete_project(project_id=19128ae8ce544f70a0fd148c537ca15b)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:31.806849 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608c7112-c23a-47cd-a46a-1504886ba13e admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:31.821639 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b96501d-a499-4b32-8697-c3dee215531b admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:31.831699 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5e3b80-bb6e-420a-bc91-9f9034c3d0c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:31.833046 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5e3b80-bb6e-420a-bc91-9f9034c3d0c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:31.833663 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5e3b80-bb6e-420a-bc91-9f9034c3d0c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:31.834713 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5e3b80-bb6e-420a-bc91-9f9034c3d0c1 admin admin] PATH_INFO: `/v3/projects/2bd43f952c704e10b43e8e205ba7e19e/users/55290c37c09f4300894051272ef602a5/roles/3771341b3be641139e752aab690397a4` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:31.860252 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5e3b80-bb6e-420a-bc91-9f9034c3d0c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bd43f952c704e10b43e8e205ba7e19e, user_id=55290c37c09f4300894051272ef602a5, role_id=3771341b3be641139e752aab690397a4)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:31.863924 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5e3b80-bb6e-420a-bc91-9f9034c3d0c1 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:31.940938 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1116/2201] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:41:31 2022] PUT /identity/v3/projects/2bd43f952c704e10b43e8e205ba7e19e/users/55290c37c09f4300894051272ef602a5/roles/3771341b3be641139e752aab690397a4 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:41:31.950154 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71a2246-b00a-4eb6-84c9-dfa671c4be2e None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:31.950927 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71a2246-b00a-4eb6-84c9-dfa671c4be2e None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:31.951396 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71a2246-b00a-4eb6-84c9-dfa671c4be2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:31.966935 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-608c7112-c23a-47cd-a46a-1504886ba13e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19128ae8ce544f70a0fd148c537ca15b', 'request_id': 'req-608c7112-c23a-47cd-a46a-1504886ba13e'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:31.993317 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-f71a2246-b00a-4eb6-84c9-dfa671c4be2e None None] MFA Rules not processed for user `55290c37c09f4300894051272ef602a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:41:31.995591 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-f71a2246-b00a-4eb6-84c9-dfa671c4be2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:41:32.031164 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-608c7112-c23a-47cd-a46a-1504886ba13e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19128ae8ce544f70a0fd148c537ca15b', 'request_id': 'req-608c7112-c23a-47cd-a46a-1504886ba13e'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:32.054692 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-608c7112-c23a-47cd-a46a-1504886ba13e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19128ae8ce544f70a0fd148c537ca15b', 'request_id': 'req-608c7112-c23a-47cd-a46a-1504886ba13e'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:32.087518 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1117/2202] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:41:31 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:41:32.092079 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71a2246-b00a-4eb6-84c9-dfa671c4be2e None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:32.098363 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-608c7112-c23a-47cd-a46a-1504886ba13e admin admin] The token cache is being invalidate because project 19128ae8ce544f70a0fd148c537ca15b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:41:32.098829 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-608c7112-c23a-47cd-a46a-1504886ba13e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:32.099830 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-608c7112-c23a-47cd-a46a-1504886ba13e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:32.104864 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1086/2203] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:41:31 2022] DELETE /identity/v3/projects/19128ae8ce544f70a0fd148c537ca15b => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:41:32.106290 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586e4337-d238-4d6c-a283-df178f6dbd7f None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:32.106588 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586e4337-d238-4d6c-a283-df178f6dbd7f None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:32.108157 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586e4337-d238-4d6c-a283-df178f6dbd7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:32.142028 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-586e4337-d238-4d6c-a283-df178f6dbd7f None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:41:32.143435 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-586e4337-d238-4d6c-a283-df178f6dbd7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:41:32.144165 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:41:32.185453 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71a2246-b00a-4eb6-84c9-dfa671c4be2e None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:32.197495 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2786fe-0c6d-46a8-99fc-5f743072a66b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:32.198296 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2786fe-0c6d-46a8-99fc-5f743072a66b admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:32.198589 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2786fe-0c6d-46a8-99fc-5f743072a66b admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:32.198894 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2786fe-0c6d-46a8-99fc-5f743072a66b admin admin] PATH_INFO: `/v3/projects` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:32.199879 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2786fe-0c6d-46a8-99fc-5f743072a66b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:32.202885 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2786fe-0c6d-46a8-99fc-5f743072a66b admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:32.236328 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1087/2204] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:41:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:41:32.246091 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2254dfe2-4434-4df3-9d9b-b47527ac62ed None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:32.246920 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2254dfe2-4434-4df3-9d9b-b47527ac62ed None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:32.247453 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2254dfe2-4434-4df3-9d9b-b47527ac62ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:32.274821 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-2254dfe2-4434-4df3-9d9b-b47527ac62ed None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:41:32.276159 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-2254dfe2-4434-4df3-9d9b-b47527ac62ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:41:32.280281 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1118/2205] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:41:32 2022] POST /identity/v3/projects => generated 391 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:41:32.284175 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2786fe-0c6d-46a8-99fc-5f743072a66b admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:32.289102 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-fc2786fe-0c6d-46a8-99fc-5f743072a66b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:41:32.289226 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1088/2206] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:41:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:41:32.292219 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2254dfe2-4434-4df3-9d9b-b47527ac62ed None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:32.304584 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2254dfe2-4434-4df3-9d9b-b47527ac62ed None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:32.377620 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:41:32.377880 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2786fe-0c6d-46a8-99fc-5f743072a66b admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:32.385064 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c365fe54-623c-4903-948f-1b37495d0382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:32.387556 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c365fe54-623c-4903-948f-1b37495d0382 admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:32.388010 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c365fe54-623c-4903-948f-1b37495d0382 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:32.388363 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c365fe54-623c-4903-948f-1b37495d0382 admin admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:32.390437 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c365fe54-623c-4903-948f-1b37495d0382 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:32.390551 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe7a79e-ed0f-481b-8062-8c91c7554250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:32.391156 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe7a79e-ed0f-481b-8062-8c91c7554250 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:32.391447 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe7a79e-ed0f-481b-8062-8c91c7554250 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:32.391731 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe7a79e-ed0f-481b-8062-8c91c7554250 admin admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:32.392586 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe7a79e-ed0f-481b-8062-8c91c7554250 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:41:32.399336 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe7a79e-ed0f-481b-8062-8c91c7554250 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:32.399477 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c365fe54-623c-4903-948f-1b37495d0382 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:32.399575 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe7a79e-ed0f-481b-8062-8c91c7554250 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:32.413794 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1089/2207] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jun 7 14:41:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:41:32.415261 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe7a79e-ed0f-481b-8062-8c91c7554250 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:32.427235 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe7a79e-ed0f-481b-8062-8c91c7554250 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:32.440308 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab7a2f2-151f-45de-8173-3ad6e3549d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:32.441532 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab7a2f2-151f-45de-8173-3ad6e3549d46 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:32.441822 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab7a2f2-151f-45de-8173-3ad6e3549d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:32.442102 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab7a2f2-151f-45de-8173-3ad6e3549d46 admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:32.442805 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab7a2f2-151f-45de-8173-3ad6e3549d46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:32.444738 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab7a2f2-151f-45de-8173-3ad6e3549d46 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:32.520042 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1119/2208] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:41:32 2022] POST /identity/v3/users => generated 302 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:41:32.522603 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c365fe54-623c-4903-948f-1b37495d0382 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:32.533761 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c365fe54-623c-4903-948f-1b37495d0382 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:32.538583 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4928067-b297-4628-91a8-15349c37ce5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:32.539261 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4928067-b297-4628-91a8-15349c37ce5a admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:32.539556 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4928067-b297-4628-91a8-15349c37ce5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:32.539840 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4928067-b297-4628-91a8-15349c37ce5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:32.541547 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4928067-b297-4628-91a8-15349c37ce5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:32.543535 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4928067-b297-4628-91a8-15349c37ce5a admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:32.552728 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1090/2209] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:41:32 2022] POST /identity/v3/projects => generated 381 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:41:32.556296 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1120/2210] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:41:32 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 14:41:32.559911 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab7a2f2-151f-45de-8173-3ad6e3549d46 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:32.560420 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4928067-b297-4628-91a8-15349c37ce5a admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:32.571566 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4928067-b297-4628-91a8-15349c37ce5a admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:32.635248 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab7a2f2-151f-45de-8173-3ad6e3549d46 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:32.636692 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7101ac2a-af25-4187-976a-9923163e3729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:32.637592 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7101ac2a-af25-4187-976a-9923163e3729 admin admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:32.637884 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7101ac2a-af25-4187-976a-9923163e3729 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:32.638221 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7101ac2a-af25-4187-976a-9923163e3729 admin admin] PATH_INFO: `/v3/projects/ddcd2d8d8e414025a334bb71a6f412e0/users/ec79617745924872b4a31159e76e0961/roles/9baff2b18f2649f7b757ff2e1f63c853` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:32.642371 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21899340-3a1b-45c1-b109-337ee132afa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:32.642921 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21899340-3a1b-45c1-b109-337ee132afa8 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:32.643740 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21899340-3a1b-45c1-b109-337ee132afa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:32.648895 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21899340-3a1b-45c1-b109-337ee132afa8 admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:32.648895 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21899340-3a1b-45c1-b109-337ee132afa8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:32.648895 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21899340-3a1b-45c1-b109-337ee132afa8 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:32.655552 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7101ac2a-af25-4187-976a-9923163e3729 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddcd2d8d8e414025a334bb71a6f412e0, user_id=ec79617745924872b4a31159e76e0961, role_id=9baff2b18f2649f7b757ff2e1f63c853)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:32.667482 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7101ac2a-af25-4187-976a-9923163e3729 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:32.740923 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1091/2211] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:41:32 2022] POST /identity/v3/users => generated 291 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:41:32.745178 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21899340-3a1b-45c1-b109-337ee132afa8 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:32.754034 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1121/2212] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:41:32 2022] PUT /identity/v3/projects/ddcd2d8d8e414025a334bb71a6f412e0/users/ec79617745924872b4a31159e76e0961/roles/9baff2b18f2649f7b757ff2e1f63c853 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:41:32.756855 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21899340-3a1b-45c1-b109-337ee132afa8 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:32.758630 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7101ac2a-af25-4187-976a-9923163e3729 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:32.779540 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:41:32.837537 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7101ac2a-af25-4187-976a-9923163e3729 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:32.840681 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0499367d-4f0e-48c1-ae9a-5df5df6dcbc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:32.841416 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0499367d-4f0e-48c1-ae9a-5df5df6dcbc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:32.841704 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0499367d-4f0e-48c1-ae9a-5df5df6dcbc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:32.841986 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0499367d-4f0e-48c1-ae9a-5df5df6dcbc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:32.842455 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a458fd-aaf0-45f1-a8a6-482660a19371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:32.842723 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0499367d-4f0e-48c1-ae9a-5df5df6dcbc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:32.843119 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a458fd-aaf0-45f1-a8a6-482660a19371 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:32.843409 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a458fd-aaf0-45f1-a8a6-482660a19371 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:32.843687 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a458fd-aaf0-45f1-a8a6-482660a19371 admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:32.844424 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0499367d-4f0e-48c1-ae9a-5df5df6dcbc9 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:32.844526 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a458fd-aaf0-45f1-a8a6-482660a19371 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:32.846052 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a458fd-aaf0-45f1-a8a6-482660a19371 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:32.855433 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1092/2213] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:41:32 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:41:32.856728 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1122/2214] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:41:32 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:41:32.859612 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0499367d-4f0e-48c1-ae9a-5df5df6dcbc9 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:32.860455 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a458fd-aaf0-45f1-a8a6-482660a19371 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:32.870504 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0499367d-4f0e-48c1-ae9a-5df5df6dcbc9 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:32.872935 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a458fd-aaf0-45f1-a8a6-482660a19371 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:32.876645 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a6ca8d-b24e-4bea-a607-57ba60c8cea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:32.876823 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4936d6d-12be-4875-a363-1ab766c33538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:32.877044 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a6ca8d-b24e-4bea-a607-57ba60c8cea0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:32.877168 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4936d6d-12be-4875-a363-1ab766c33538 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:32.877326 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a6ca8d-b24e-4bea-a607-57ba60c8cea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:32.877421 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4936d6d-12be-4875-a363-1ab766c33538 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:32.877612 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a6ca8d-b24e-4bea-a607-57ba60c8cea0 admin admin] PATH_INFO: `/v3/projects/ddcd2d8d8e414025a334bb71a6f412e0/users/ec79617745924872b4a31159e76e0961/roles/9baff2b18f2649f7b757ff2e1f63c853` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:32.877748 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4936d6d-12be-4875-a363-1ab766c33538 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:32.879312 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4936d6d-12be-4875-a363-1ab766c33538 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:32.880261 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a6ca8d-b24e-4bea-a607-57ba60c8cea0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddcd2d8d8e414025a334bb71a6f412e0, user_id=ec79617745924872b4a31159e76e0961, role_id=9baff2b18f2649f7b757ff2e1f63c853)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:32.881256 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4936d6d-12be-4875-a363-1ab766c33538 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:32.885234 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a6ca8d-b24e-4bea-a607-57ba60c8cea0 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:32.892985 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1093/2215] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:41:32 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:41:32.897148 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4936d6d-12be-4875-a363-1ab766c33538 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:32.898171 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1123/2216] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:41:32 2022] PUT /identity/v3/projects/ddcd2d8d8e414025a334bb71a6f412e0/users/ec79617745924872b4a31159e76e0961/roles/9baff2b18f2649f7b757ff2e1f63c853 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:41:32.908853 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f73a816-596e-4295-8ccd-46ae514a9e72 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:32.916148 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f73a816-596e-4295-8ccd-46ae514a9e72 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:32.916148 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f73a816-596e-4295-8ccd-46ae514a9e72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:32.916404 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4936d6d-12be-4875-a363-1ab766c33538 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:32.937593 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-5f73a816-596e-4295-8ccd-46ae514a9e72 None None] MFA Rules not processed for user `ec79617745924872b4a31159e76e0961`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:41:32.939051 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-5f73a816-596e-4295-8ccd-46ae514a9e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:41:32.953950 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:41:32.980435 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2663e9-03df-4c20-9ead-ac0b684e16c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:32.981336 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2663e9-03df-4c20-9ead-ac0b684e16c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:32.981619 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2663e9-03df-4c20-9ead-ac0b684e16c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:32.981899 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2663e9-03df-4c20-9ead-ac0b684e16c8 admin admin] PATH_INFO: `/v3/projects/e434d85936764f06b84bf42f61d68369/users/536cb3d0da444d2dbeddf5db9e4f20b1/roles/3771341b3be641139e752aab690397a4` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:32.997643 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2663e9-03df-4c20-9ead-ac0b684e16c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e434d85936764f06b84bf42f61d68369, user_id=536cb3d0da444d2dbeddf5db9e4f20b1, role_id=3771341b3be641139e752aab690397a4)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:33.000438 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2663e9-03df-4c20-9ead-ac0b684e16c8 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:33.014704 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1124/2217] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:41:32 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:41:33.026273 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f73a816-596e-4295-8ccd-46ae514a9e72 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:33.027723 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-5f73a816-596e-4295-8ccd-46ae514a9e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:41:33.087060 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f73a816-596e-4295-8ccd-46ae514a9e72 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:33.090939 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78973a56-810a-4848-99fc-e3815c706715 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d81f597cb44c56a966537089bcce4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:33.091623 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78973a56-810a-4848-99fc-e3815c706715 service cinder] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:33.091896 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78973a56-810a-4848-99fc-e3815c706715 service cinder] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:33.092185 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78973a56-810a-4848-99fc-e3815c706715 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:33.097696 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1094/2218] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:41:32 2022] PUT /identity/v3/projects/e434d85936764f06b84bf42f61d68369/users/536cb3d0da444d2dbeddf5db9e4f20b1/roles/3771341b3be641139e752aab690397a4 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:41:33.102642 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78973a56-810a-4848-99fc-e3815c706715 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:33.104171 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78973a56-810a-4848-99fc-e3815c706715 service cinder] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:33.108145 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c684d61d-6349-4e56-9402-7bbd7736ff53 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:33.108469 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c684d61d-6349-4e56-9402-7bbd7736ff53 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:33.108751 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c684d61d-6349-4e56-9402-7bbd7736ff53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:33.115376 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1125/2219] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:41:33 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:41:33.142019 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-c684d61d-6349-4e56-9402-7bbd7736ff53 None None] MFA Rules not processed for user `536cb3d0da444d2dbeddf5db9e4f20b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:41:33.143403 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-c684d61d-6349-4e56-9402-7bbd7736ff53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:41:33.167667 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78973a56-810a-4848-99fc-e3815c706715 service cinder] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:33.169483 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-78973a56-810a-4848-99fc-e3815c706715 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:41:33.191403 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78973a56-810a-4848-99fc-e3815c706715 service cinder] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:33.214405 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1095/2220] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:41:33 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:41:33.222646 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:41:33.226551 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c684d61d-6349-4e56-9402-7bbd7736ff53 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:33.246874 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dfea42-8a4b-4c5f-8c56-c9cf62025719 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d81f597cb44c56a966537089bcce4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:33.247634 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dfea42-8a4b-4c5f-8c56-c9cf62025719 service cinder] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:33.248049 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dfea42-8a4b-4c5f-8c56-c9cf62025719 service cinder] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:33.248477 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dfea42-8a4b-4c5f-8c56-c9cf62025719 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:33.250388 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-e8dfea42-8a4b-4c5f-8c56-c9cf62025719 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:41:33.284625 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c684d61d-6349-4e56-9402-7bbd7736ff53 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:33.289174 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d985cf3-d092-4425-8cc5-97ad5e14e320 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d81f597cb44c56a966537089bcce4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:33.289828 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d985cf3-d092-4425-8cc5-97ad5e14e320 service cinder] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:33.290115 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d985cf3-d092-4425-8cc5-97ad5e14e320 service cinder] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:33.290397 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d985cf3-d092-4425-8cc5-97ad5e14e320 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:33.299075 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d985cf3-d092-4425-8cc5-97ad5e14e320 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:33.300692 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d985cf3-d092-4425-8cc5-97ad5e14e320 service cinder] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:33.309942 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1096/2221] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:41:33 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 14:41:33.314680 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dfea42-8a4b-4c5f-8c56-c9cf62025719 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:33.316753 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dfea42-8a4b-4c5f-8c56-c9cf62025719 service cinder] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:33.327123 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1126/2222] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:41:33 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 14:41:36.958046 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d985cf3-d092-4425-8cc5-97ad5e14e320 service cinder] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:36.960046 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-0d985cf3-d092-4425-8cc5-97ad5e14e320 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:41:37.020436 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:41:37.027046 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d985cf3-d092-4425-8cc5-97ad5e14e320 service cinder] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:37.031827 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626ccfc3-0b1b-43a1-ba94-b5971367ec9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:37.035513 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626ccfc3-0b1b-43a1-ba94-b5971367ec9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:37.035513 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626ccfc3-0b1b-43a1-ba94-b5971367ec9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:37.035513 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626ccfc3-0b1b-43a1-ba94-b5971367ec9e admin admin] PATH_INFO: `/v3/users/c813f52639e24cec86b64eeae4d99b60` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:37.035513 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626ccfc3-0b1b-43a1-ba94-b5971367ec9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c813f52639e24cec86b64eeae4d99b60)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:37.036506 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626ccfc3-0b1b-43a1-ba94-b5971367ec9e admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:37.275793 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-626ccfc3-0b1b-43a1-ba94-b5971367ec9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c813f52639e24cec86b64eeae4d99b60'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:37.277081 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-626ccfc3-0b1b-43a1-ba94-b5971367ec9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c813f52639e24cec86b64eeae4d99b60'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:37.279229 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-626ccfc3-0b1b-43a1-ba94-b5971367ec9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c813f52639e24cec86b64eeae4d99b60'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:37.289180 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-626ccfc3-0b1b-43a1-ba94-b5971367ec9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c813f52639e24cec86b64eeae4d99b60'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:37.302919 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-626ccfc3-0b1b-43a1-ba94-b5971367ec9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c813f52639e24cec86b64eeae4d99b60'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:37.387466 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1097/2223] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:41:36 2022] DELETE /identity/v3/users/c813f52639e24cec86b64eeae4d99b60 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:41:37.394414 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770395da-ee55-48b9-b858-ffa7994be330 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:37.394890 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770395da-ee55-48b9-b858-ffa7994be330 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:37.395068 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770395da-ee55-48b9-b858-ffa7994be330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:37.414615 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-770395da-ee55-48b9-b858-ffa7994be330 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:41:37.415835 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-770395da-ee55-48b9-b858-ffa7994be330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:41:37.477632 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1127/2224] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:41:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:41:37.487277 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626ccfc3-0b1b-43a1-ba94-b5971367ec9e admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:37.488825 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-626ccfc3-0b1b-43a1-ba94-b5971367ec9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:41:37.551905 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626ccfc3-0b1b-43a1-ba94-b5971367ec9e admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:37.555842 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464e7c0d-9082-4bc0-8751-56f5de78b8e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:37.556527 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464e7c0d-9082-4bc0-8751-56f5de78b8e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:37.556832 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464e7c0d-9082-4bc0-8751-56f5de78b8e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:37.557083 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464e7c0d-9082-4bc0-8751-56f5de78b8e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:37.566957 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464e7c0d-9082-4bc0-8751-56f5de78b8e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:37.568399 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464e7c0d-9082-4bc0-8751-56f5de78b8e4 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:37.579030 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1098/2225] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:41:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:41:37.613782 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770395da-ee55-48b9-b858-ffa7994be330 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:37.616673 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-770395da-ee55-48b9-b858-ffa7994be330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:41:37.641921 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770395da-ee55-48b9-b858-ffa7994be330 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:37.648515 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422ac1be-bb50-4bc8-b94c-1097b918869b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:37.648975 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422ac1be-bb50-4bc8-b94c-1097b918869b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:37.649263 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422ac1be-bb50-4bc8-b94c-1097b918869b admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:37.649543 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422ac1be-bb50-4bc8-b94c-1097b918869b admin admin] PATH_INFO: `/v3/users/536cb3d0da444d2dbeddf5db9e4f20b1` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:37.650781 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422ac1be-bb50-4bc8-b94c-1097b918869b admin admin] RBAC: Authorizing `identity:delete_user(user_id=536cb3d0da444d2dbeddf5db9e4f20b1)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:37.652851 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422ac1be-bb50-4bc8-b94c-1097b918869b admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:37.877329 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-422ac1be-bb50-4bc8-b94c-1097b918869b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '536cb3d0da444d2dbeddf5db9e4f20b1'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:37.878282 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-422ac1be-bb50-4bc8-b94c-1097b918869b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '536cb3d0da444d2dbeddf5db9e4f20b1'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:37.878897 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-422ac1be-bb50-4bc8-b94c-1097b918869b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '536cb3d0da444d2dbeddf5db9e4f20b1'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:37.889527 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-422ac1be-bb50-4bc8-b94c-1097b918869b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '536cb3d0da444d2dbeddf5db9e4f20b1'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:37.906870 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-422ac1be-bb50-4bc8-b94c-1097b918869b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '536cb3d0da444d2dbeddf5db9e4f20b1'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:37.911120 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:41:38.010964 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1128/2226] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:41:37 2022] DELETE /identity/v3/users/536cb3d0da444d2dbeddf5db9e4f20b1 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:41:38.015940 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54132ee1-1401-400a-a4fa-5b1c66b07421 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:38.016532 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54132ee1-1401-400a-a4fa-5b1c66b07421 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:38.016686 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54132ee1-1401-400a-a4fa-5b1c66b07421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:38.034436 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-54132ee1-1401-400a-a4fa-5b1c66b07421 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:41:38.035627 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-54132ee1-1401-400a-a4fa-5b1c66b07421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:41:38.095668 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422ac1be-bb50-4bc8-b94c-1097b918869b admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:38.098416 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-422ac1be-bb50-4bc8-b94c-1097b918869b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:41:38.100741 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1099/2227] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:41:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:41:38.111259 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54132ee1-1401-400a-a4fa-5b1c66b07421 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:38.112793 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-54132ee1-1401-400a-a4fa-5b1c66b07421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:41:38.117234 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422ac1be-bb50-4bc8-b94c-1097b918869b admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:38.124503 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207199ef-d67c-44e6-881f-68488a387732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:38.125631 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207199ef-d67c-44e6-881f-68488a387732 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:38.125921 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207199ef-d67c-44e6-881f-68488a387732 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:38.126200 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207199ef-d67c-44e6-881f-68488a387732 admin admin] PATH_INFO: `/v3/projects/39ef8e106ba4454aab0ae9410aa51f0d` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:38.127596 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207199ef-d67c-44e6-881f-68488a387732 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39ef8e106ba4454aab0ae9410aa51f0d)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:38.130485 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207199ef-d67c-44e6-881f-68488a387732 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:38.136821 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:41:38.178612 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54132ee1-1401-400a-a4fa-5b1c66b07421 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:38.182709 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb8d2ef-3e1e-4fcc-a59a-3cfb1e4e2f86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:38.183460 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb8d2ef-3e1e-4fcc-a59a-3cfb1e4e2f86 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:38.183677 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb8d2ef-3e1e-4fcc-a59a-3cfb1e4e2f86 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:38.183956 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb8d2ef-3e1e-4fcc-a59a-3cfb1e4e2f86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:38.197356 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb8d2ef-3e1e-4fcc-a59a-3cfb1e4e2f86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:38.198801 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb8d2ef-3e1e-4fcc-a59a-3cfb1e4e2f86 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:38.209427 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1100/2228] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:41:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 14:41:38.267601 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-207199ef-d67c-44e6-881f-68488a387732 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39ef8e106ba4454aab0ae9410aa51f0d', 'request_id': 'req-207199ef-d67c-44e6-881f-68488a387732'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:38.373940 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-207199ef-d67c-44e6-881f-68488a387732 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39ef8e106ba4454aab0ae9410aa51f0d', 'request_id': 'req-207199ef-d67c-44e6-881f-68488a387732'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:38.380150 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-207199ef-d67c-44e6-881f-68488a387732 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39ef8e106ba4454aab0ae9410aa51f0d', 'request_id': 'req-207199ef-d67c-44e6-881f-68488a387732'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:38.393025 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-207199ef-d67c-44e6-881f-68488a387732 admin admin] The token cache is being invalidate because project 39ef8e106ba4454aab0ae9410aa51f0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:41:38.393323 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-207199ef-d67c-44e6-881f-68488a387732 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:38.394093 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-207199ef-d67c-44e6-881f-68488a387732 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:38.395592 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1129/2229] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:41:38 2022] DELETE /identity/v3/projects/39ef8e106ba4454aab0ae9410aa51f0d => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:41:38.399790 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb8d2ef-3e1e-4fcc-a59a-3cfb1e4e2f86 service neutron] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:38.409576 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb8d2ef-3e1e-4fcc-a59a-3cfb1e4e2f86 service neutron] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:38.479656 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cc7973-783f-4213-834c-26220a62a84e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:38.481054 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cc7973-783f-4213-834c-26220a62a84e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:38.481054 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cc7973-783f-4213-834c-26220a62a84e admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:38.481481 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cc7973-783f-4213-834c-26220a62a84e admin admin] PATH_INFO: `/v3/users/db5101ff94444707a2fa0548bd92b818` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:38.483072 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cc7973-783f-4213-834c-26220a62a84e admin admin] RBAC: Authorizing `identity:delete_user(user_id=db5101ff94444707a2fa0548bd92b818)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:38.485255 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cc7973-783f-4213-834c-26220a62a84e admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:38.668404 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207199ef-d67c-44e6-881f-68488a387732 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:38.669926 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-207199ef-d67c-44e6-881f-68488a387732 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:41:38.699327 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207199ef-d67c-44e6-881f-68488a387732 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:38.704225 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a059cad7-ad41-4dbd-9a45-7e6a4e1257c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:38.705147 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a059cad7-ad41-4dbd-9a45-7e6a4e1257c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:38.705261 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-98cc7973-783f-4213-834c-26220a62a84e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db5101ff94444707a2fa0548bd92b818'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:38.705435 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a059cad7-ad41-4dbd-9a45-7e6a4e1257c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:38.705710 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a059cad7-ad41-4dbd-9a45-7e6a4e1257c5 admin admin] PATH_INFO: `/v3/projects/e434d85936764f06b84bf42f61d68369` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:38.705922 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-98cc7973-783f-4213-834c-26220a62a84e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db5101ff94444707a2fa0548bd92b818'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:38.706531 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-98cc7973-783f-4213-834c-26220a62a84e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db5101ff94444707a2fa0548bd92b818'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:38.707003 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a059cad7-ad41-4dbd-9a45-7e6a4e1257c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e434d85936764f06b84bf42f61d68369)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:38.708944 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a059cad7-ad41-4dbd-9a45-7e6a4e1257c5 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:38.714396 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-98cc7973-783f-4213-834c-26220a62a84e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db5101ff94444707a2fa0548bd92b818'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:38.728655 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-98cc7973-783f-4213-834c-26220a62a84e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db5101ff94444707a2fa0548bd92b818'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:38.855152 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1101/2230] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:41:38 2022] DELETE /identity/v3/users/db5101ff94444707a2fa0548bd92b818 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:41:38.926576 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-a059cad7-ad41-4dbd-9a45-7e6a4e1257c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e434d85936764f06b84bf42f61d68369', 'request_id': 'req-a059cad7-ad41-4dbd-9a45-7e6a4e1257c5'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:39.019795 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-a059cad7-ad41-4dbd-9a45-7e6a4e1257c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e434d85936764f06b84bf42f61d68369', 'request_id': 'req-a059cad7-ad41-4dbd-9a45-7e6a4e1257c5'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:39.027440 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-a059cad7-ad41-4dbd-9a45-7e6a4e1257c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e434d85936764f06b84bf42f61d68369', 'request_id': 'req-a059cad7-ad41-4dbd-9a45-7e6a4e1257c5'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:39.040265 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-a059cad7-ad41-4dbd-9a45-7e6a4e1257c5 admin admin] The token cache is being invalidate because project e434d85936764f06b84bf42f61d68369 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:41:39.040595 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-a059cad7-ad41-4dbd-9a45-7e6a4e1257c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:39.041671 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-a059cad7-ad41-4dbd-9a45-7e6a4e1257c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:39.043248 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1130/2231] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:41:38 2022] DELETE /identity/v3/projects/e434d85936764f06b84bf42f61d68369 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:41:39.335451 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cc7973-783f-4213-834c-26220a62a84e admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:39.337170 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-98cc7973-783f-4213-834c-26220a62a84e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:41:39.383299 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:41:39.404445 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cc7973-783f-4213-834c-26220a62a84e admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:39.461495 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45dfd9b-59e7-4ea0-b0e1-1378944eacbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:39.462365 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45dfd9b-59e7-4ea0-b0e1-1378944eacbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:39.462655 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45dfd9b-59e7-4ea0-b0e1-1378944eacbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:39.462927 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45dfd9b-59e7-4ea0-b0e1-1378944eacbb admin admin] PATH_INFO: `/v3/projects/03026ac1e53f4518b19f91bf7fa954cc` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:39.464108 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45dfd9b-59e7-4ea0-b0e1-1378944eacbb admin admin] RBAC: Authorizing `identity:delete_project(project_id=03026ac1e53f4518b19f91bf7fa954cc)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:39.466062 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45dfd9b-59e7-4ea0-b0e1-1378944eacbb admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:39.586803 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a45dfd9b-59e7-4ea0-b0e1-1378944eacbb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03026ac1e53f4518b19f91bf7fa954cc', 'request_id': 'req-a45dfd9b-59e7-4ea0-b0e1-1378944eacbb'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:39.701104 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a45dfd9b-59e7-4ea0-b0e1-1378944eacbb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03026ac1e53f4518b19f91bf7fa954cc', 'request_id': 'req-a45dfd9b-59e7-4ea0-b0e1-1378944eacbb'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:39.710144 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a45dfd9b-59e7-4ea0-b0e1-1378944eacbb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03026ac1e53f4518b19f91bf7fa954cc', 'request_id': 'req-a45dfd9b-59e7-4ea0-b0e1-1378944eacbb'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:39.730317 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a45dfd9b-59e7-4ea0-b0e1-1378944eacbb admin admin] The token cache is being invalidate because project 03026ac1e53f4518b19f91bf7fa954cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:41:39.730714 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a45dfd9b-59e7-4ea0-b0e1-1378944eacbb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:39.731962 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a45dfd9b-59e7-4ea0-b0e1-1378944eacbb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:39.736416 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1102/2232] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:41:39 2022] DELETE /identity/v3/projects/03026ac1e53f4518b19f91bf7fa954cc => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:41:39.738680 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a059cad7-ad41-4dbd-9a45-7e6a4e1257c5 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:39.740825 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-a059cad7-ad41-4dbd-9a45-7e6a4e1257c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:41:39.754510 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a059cad7-ad41-4dbd-9a45-7e6a4e1257c5 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:39.818417 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f5ea8b-6e99-4d46-8bd4-6fe778f848fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:39.819916 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f5ea8b-6e99-4d46-8bd4-6fe778f848fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:39.820210 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f5ea8b-6e99-4d46-8bd4-6fe778f848fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:39.820521 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f5ea8b-6e99-4d46-8bd4-6fe778f848fe admin admin] PATH_INFO: `/v3/users/481331f1597e4076bc16bbe05e5060d5` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:39.821783 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f5ea8b-6e99-4d46-8bd4-6fe778f848fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=481331f1597e4076bc16bbe05e5060d5)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:39.823899 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f5ea8b-6e99-4d46-8bd4-6fe778f848fe admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:39.825732 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:41:40.030998 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-98f5ea8b-6e99-4d46-8bd4-6fe778f848fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '481331f1597e4076bc16bbe05e5060d5'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:40.031812 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-98f5ea8b-6e99-4d46-8bd4-6fe778f848fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '481331f1597e4076bc16bbe05e5060d5'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:40.032542 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-98f5ea8b-6e99-4d46-8bd4-6fe778f848fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '481331f1597e4076bc16bbe05e5060d5'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:40.042406 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-98f5ea8b-6e99-4d46-8bd4-6fe778f848fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '481331f1597e4076bc16bbe05e5060d5'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:40.057412 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-98f5ea8b-6e99-4d46-8bd4-6fe778f848fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '481331f1597e4076bc16bbe05e5060d5'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:40.152821 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1131/2233] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:41:39 2022] DELETE /identity/v3/users/481331f1597e4076bc16bbe05e5060d5 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:41:40.593589 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45dfd9b-59e7-4ea0-b0e1-1378944eacbb admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:40.595088 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-a45dfd9b-59e7-4ea0-b0e1-1378944eacbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:41:40.607909 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45dfd9b-59e7-4ea0-b0e1-1378944eacbb admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:40.653996 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f5ea8b-6e99-4d46-8bd4-6fe778f848fe admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:40.655718 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-98f5ea8b-6e99-4d46-8bd4-6fe778f848fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:41:40.677045 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbb3b36-0b58-4988-977d-62997db2c22c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:40.677929 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbb3b36-0b58-4988-977d-62997db2c22c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:40.678256 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbb3b36-0b58-4988-977d-62997db2c22c admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:40.680505 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbb3b36-0b58-4988-977d-62997db2c22c admin admin] PATH_INFO: `/v3/projects/e3d47631ae6f46f3993da856993fe000` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:40.680505 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbb3b36-0b58-4988-977d-62997db2c22c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3d47631ae6f46f3993da856993fe000)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:40.681847 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbb3b36-0b58-4988-977d-62997db2c22c admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:40.704542 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:41:40.722029 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f5ea8b-6e99-4d46-8bd4-6fe778f848fe admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:40.727435 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50ff5c8-91c7-48c8-84f6-50805b966548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:40.728321 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50ff5c8-91c7-48c8-84f6-50805b966548 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:40.728637 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50ff5c8-91c7-48c8-84f6-50805b966548 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:40.728914 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50ff5c8-91c7-48c8-84f6-50805b966548 admin admin] PATH_INFO: `/v3/users/55290c37c09f4300894051272ef602a5` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:40.730153 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50ff5c8-91c7-48c8-84f6-50805b966548 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55290c37c09f4300894051272ef602a5)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:40.732318 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50ff5c8-91c7-48c8-84f6-50805b966548 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:40.797031 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-7cbb3b36-0b58-4988-977d-62997db2c22c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3d47631ae6f46f3993da856993fe000', 'request_id': 'req-7cbb3b36-0b58-4988-977d-62997db2c22c'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:40.911649 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-7cbb3b36-0b58-4988-977d-62997db2c22c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3d47631ae6f46f3993da856993fe000', 'request_id': 'req-7cbb3b36-0b58-4988-977d-62997db2c22c'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:40.918028 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-7cbb3b36-0b58-4988-977d-62997db2c22c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3d47631ae6f46f3993da856993fe000', 'request_id': 'req-7cbb3b36-0b58-4988-977d-62997db2c22c'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:40.929805 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-7cbb3b36-0b58-4988-977d-62997db2c22c admin admin] The token cache is being invalidate because project e3d47631ae6f46f3993da856993fe000 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:41:40.930096 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-7cbb3b36-0b58-4988-977d-62997db2c22c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:40.930745 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-7cbb3b36-0b58-4988-977d-62997db2c22c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:40.932294 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1103/2234] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:41:40 2022] DELETE /identity/v3/projects/e3d47631ae6f46f3993da856993fe000 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:41:41.013587 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-f50ff5c8-91c7-48c8-84f6-50805b966548 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55290c37c09f4300894051272ef602a5'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:41.014553 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-f50ff5c8-91c7-48c8-84f6-50805b966548 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55290c37c09f4300894051272ef602a5'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:41.015466 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-f50ff5c8-91c7-48c8-84f6-50805b966548 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55290c37c09f4300894051272ef602a5'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:41.023739 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-f50ff5c8-91c7-48c8-84f6-50805b966548 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55290c37c09f4300894051272ef602a5'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:41.038224 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-f50ff5c8-91c7-48c8-84f6-50805b966548 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55290c37c09f4300894051272ef602a5'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:41.101908 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:41:41.107744 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1132/2235] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:41:40 2022] DELETE /identity/v3/users/55290c37c09f4300894051272ef602a5 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:41:41.113073 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0731728-e28a-4383-b442-c502299a12a5 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:41.113364 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0731728-e28a-4383-b442-c502299a12a5 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:41.113642 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0731728-e28a-4383-b442-c502299a12a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:41.135645 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-b0731728-e28a-4383-b442-c502299a12a5 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:41:41.137051 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-b0731728-e28a-4383-b442-c502299a12a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:41:41.205395 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1104/2236] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:41:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:41:41.215080 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50ff5c8-91c7-48c8-84f6-50805b966548 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:41.216554 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-f50ff5c8-91c7-48c8-84f6-50805b966548 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:41:41.275664 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50ff5c8-91c7-48c8-84f6-50805b966548 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:41.279352 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c88b160-0709-4977-8e19-fadc155e56b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:41.279999 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c88b160-0709-4977-8e19-fadc155e56b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:41.280284 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c88b160-0709-4977-8e19-fadc155e56b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:41.280593 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c88b160-0709-4977-8e19-fadc155e56b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:41.290232 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c88b160-0709-4977-8e19-fadc155e56b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:41.291667 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c88b160-0709-4977-8e19-fadc155e56b4 service neutron] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:41.301490 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1133/2237] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:41:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:41:41.702833 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0731728-e28a-4383-b442-c502299a12a5 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:41.704656 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-b0731728-e28a-4383-b442-c502299a12a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:41:41.719725 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0731728-e28a-4383-b442-c502299a12a5 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:41.725382 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07925673-022a-4b43-b646-8485a5279d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:41.726621 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07925673-022a-4b43-b646-8485a5279d89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:41.727017 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07925673-022a-4b43-b646-8485a5279d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:41.727216 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07925673-022a-4b43-b646-8485a5279d89 admin admin] PATH_INFO: `/v3/projects/2bd43f952c704e10b43e8e205ba7e19e` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:41.728621 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07925673-022a-4b43-b646-8485a5279d89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bd43f952c704e10b43e8e205ba7e19e)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:41.730848 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07925673-022a-4b43-b646-8485a5279d89 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:41.842277 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-07925673-022a-4b43-b646-8485a5279d89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bd43f952c704e10b43e8e205ba7e19e', 'request_id': 'req-07925673-022a-4b43-b646-8485a5279d89'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:41.905098 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-07925673-022a-4b43-b646-8485a5279d89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bd43f952c704e10b43e8e205ba7e19e', 'request_id': 'req-07925673-022a-4b43-b646-8485a5279d89'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:41.911112 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-07925673-022a-4b43-b646-8485a5279d89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bd43f952c704e10b43e8e205ba7e19e', 'request_id': 'req-07925673-022a-4b43-b646-8485a5279d89'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:41.922894 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-07925673-022a-4b43-b646-8485a5279d89 admin admin] The token cache is being invalidate because project 2bd43f952c704e10b43e8e205ba7e19e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:41:41.923061 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-07925673-022a-4b43-b646-8485a5279d89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:41.923707 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-07925673-022a-4b43-b646-8485a5279d89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:41.925605 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1105/2238] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:41:41 2022] DELETE /identity/v3/projects/2bd43f952c704e10b43e8e205ba7e19e => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:41:41.929937 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c88b160-0709-4977-8e19-fadc155e56b4 service neutron] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:41.939675 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c88b160-0709-4977-8e19-fadc155e56b4 service neutron] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:42.002034 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa13c6b-bdbe-4757-ae47-8cd6b4f4ddaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:42.002921 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa13c6b-bdbe-4757-ae47-8cd6b4f4ddaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:42.003212 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa13c6b-bdbe-4757-ae47-8cd6b4f4ddaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:42.003499 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa13c6b-bdbe-4757-ae47-8cd6b4f4ddaa admin admin] PATH_INFO: `/v3/users/ec79617745924872b4a31159e76e0961` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:42.004741 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa13c6b-bdbe-4757-ae47-8cd6b4f4ddaa admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec79617745924872b4a31159e76e0961)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:42.006757 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa13c6b-bdbe-4757-ae47-8cd6b4f4ddaa admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:42.199114 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-faa13c6b-bdbe-4757-ae47-8cd6b4f4ddaa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec79617745924872b4a31159e76e0961'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:42.199777 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-faa13c6b-bdbe-4757-ae47-8cd6b4f4ddaa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec79617745924872b4a31159e76e0961'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:42.200352 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-faa13c6b-bdbe-4757-ae47-8cd6b4f4ddaa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec79617745924872b4a31159e76e0961'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:42.207041 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-faa13c6b-bdbe-4757-ae47-8cd6b4f4ddaa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec79617745924872b4a31159e76e0961'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:42.219681 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-faa13c6b-bdbe-4757-ae47-8cd6b4f4ddaa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec79617745924872b4a31159e76e0961'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:42.288172 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1134/2239] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:41:41 2022] DELETE /identity/v3/users/ec79617745924872b4a31159e76e0961 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:41:42.689351 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07925673-022a-4b43-b646-8485a5279d89 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:42.690849 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-07925673-022a-4b43-b646-8485a5279d89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:41:42.709208 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07925673-022a-4b43-b646-8485a5279d89 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:42.745428 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:41:42.773213 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad416734-18e3-47cf-9a5b-8e1ee01adeb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:42.773930 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad416734-18e3-47cf-9a5b-8e1ee01adeb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:42.774234 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad416734-18e3-47cf-9a5b-8e1ee01adeb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:42.774513 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad416734-18e3-47cf-9a5b-8e1ee01adeb8 admin admin] PATH_INFO: `/v3/projects/ddcd2d8d8e414025a334bb71a6f412e0` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:42.775713 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad416734-18e3-47cf-9a5b-8e1ee01adeb8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ddcd2d8d8e414025a334bb71a6f412e0)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:42.777682 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad416734-18e3-47cf-9a5b-8e1ee01adeb8 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:42.952357 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-ad416734-18e3-47cf-9a5b-8e1ee01adeb8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ddcd2d8d8e414025a334bb71a6f412e0', 'request_id': 'req-ad416734-18e3-47cf-9a5b-8e1ee01adeb8'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:43.025369 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-ad416734-18e3-47cf-9a5b-8e1ee01adeb8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ddcd2d8d8e414025a334bb71a6f412e0', 'request_id': 'req-ad416734-18e3-47cf-9a5b-8e1ee01adeb8'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:43.031632 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-ad416734-18e3-47cf-9a5b-8e1ee01adeb8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ddcd2d8d8e414025a334bb71a6f412e0', 'request_id': 'req-ad416734-18e3-47cf-9a5b-8e1ee01adeb8'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:43.042979 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-ad416734-18e3-47cf-9a5b-8e1ee01adeb8 admin admin] The token cache is being invalidate because project ddcd2d8d8e414025a334bb71a6f412e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:41:43.043271 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-ad416734-18e3-47cf-9a5b-8e1ee01adeb8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:43.043913 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-ad416734-18e3-47cf-9a5b-8e1ee01adeb8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:41:43.045585 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1106/2240] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:41:42 2022] DELETE /identity/v3/projects/ddcd2d8d8e414025a334bb71a6f412e0 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:41:43.051365 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2489c6be-2575-493f-a56a-7e813bbdb29f None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:43.051668 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2489c6be-2575-493f-a56a-7e813bbdb29f None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:43.051955 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2489c6be-2575-493f-a56a-7e813bbdb29f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:43.071156 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-2489c6be-2575-493f-a56a-7e813bbdb29f None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:41:43.072383 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-2489c6be-2575-493f-a56a-7e813bbdb29f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:41:43.090673 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:41:43.134874 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1135/2241] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:41:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:41:43.142883 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e845489e-35b7-49fd-93d9-58035d51622e None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:43.143174 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e845489e-35b7-49fd-93d9-58035d51622e None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:43.143451 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e845489e-35b7-49fd-93d9-58035d51622e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:43.161520 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-e845489e-35b7-49fd-93d9-58035d51622e None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:41:43.162716 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-e845489e-35b7-49fd-93d9-58035d51622e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:41:43.170776 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1107/2242] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:41:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:41:43.175113 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2489c6be-2575-493f-a56a-7e813bbdb29f None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:43.185340 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2489c6be-2575-493f-a56a-7e813bbdb29f None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:43.190180 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be406b50-87c5-434c-9318-ed50815ae4ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:43.190844 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be406b50-87c5-434c-9318-ed50815ae4ea admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:43.191136 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be406b50-87c5-434c-9318-ed50815ae4ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:43.191421 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be406b50-87c5-434c-9318-ed50815ae4ea admin admin] PATH_INFO: `/v3/domains` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:43.192190 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be406b50-87c5-434c-9318-ed50815ae4ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:41:43.192535 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be406b50-87c5-434c-9318-ed50815ae4ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:43.194722 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be406b50-87c5-434c-9318-ed50815ae4ea admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:43.207461 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1136/2243] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jun 7 14:41:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:41:43.215306 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e845489e-35b7-49fd-93d9-58035d51622e None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:43.221964 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e845489e-35b7-49fd-93d9-58035d51622e None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:43.227707 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d715a3b-1348-4b7a-a793-81d5df8f3624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:43.228064 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d715a3b-1348-4b7a-a793-81d5df8f3624 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:43.228352 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d715a3b-1348-4b7a-a793-81d5df8f3624 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:43.228662 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d715a3b-1348-4b7a-a793-81d5df8f3624 admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:43.229572 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d715a3b-1348-4b7a-a793-81d5df8f3624 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:43.231498 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d715a3b-1348-4b7a-a793-81d5df8f3624 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:43.312895 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1108/2244] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:41:43 2022] POST /identity/v3/projects => generated 371 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:41:43.316894 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be406b50-87c5-434c-9318-ed50815ae4ea admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:43.327370 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be406b50-87c5-434c-9318-ed50815ae4ea admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:43.382915 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ed4555-ddc4-401b-8306-d6ba3e7118c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:43.383906 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ed4555-ddc4-401b-8306-d6ba3e7118c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:43.384191 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ed4555-ddc4-401b-8306-d6ba3e7118c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:43.384514 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ed4555-ddc4-401b-8306-d6ba3e7118c9 admin admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:43.385215 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ed4555-ddc4-401b-8306-d6ba3e7118c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:43.387139 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ed4555-ddc4-401b-8306-d6ba3e7118c9 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:43.488563 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1137/2245] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:41:43 2022] POST /identity/v3/users => generated 286 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:41:43.493304 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d715a3b-1348-4b7a-a793-81d5df8f3624 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:43.503996 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d715a3b-1348-4b7a-a793-81d5df8f3624 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:43.508754 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff32bfcb-957b-4083-8e14-ea3526d11b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:43.509449 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff32bfcb-957b-4083-8e14-ea3526d11b5b admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:43.509755 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff32bfcb-957b-4083-8e14-ea3526d11b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:43.510033 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff32bfcb-957b-4083-8e14-ea3526d11b5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:43.510910 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff32bfcb-957b-4083-8e14-ea3526d11b5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:43.512671 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff32bfcb-957b-4083-8e14-ea3526d11b5b admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:43.523283 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1109/2246] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:41:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:41:43.527341 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ed4555-ddc4-401b-8306-d6ba3e7118c9 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:43.538647 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ed4555-ddc4-401b-8306-d6ba3e7118c9 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:43.549022 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da5aac1-7d95-40c3-b5be-d5adb6f834df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:43.549711 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da5aac1-7d95-40c3-b5be-d5adb6f834df admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:43.549992 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da5aac1-7d95-40c3-b5be-d5adb6f834df admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:43.550268 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da5aac1-7d95-40c3-b5be-d5adb6f834df admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:43.551001 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da5aac1-7d95-40c3-b5be-d5adb6f834df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:43.552637 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da5aac1-7d95-40c3-b5be-d5adb6f834df admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:43.563873 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1138/2247] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:41:43 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:41:43.567993 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff32bfcb-957b-4083-8e14-ea3526d11b5b admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:43.577996 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff32bfcb-957b-4083-8e14-ea3526d11b5b admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:43.582627 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa91960-c5d2-439d-bfa2-a97b2cc4571c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:43.583505 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa91960-c5d2-439d-bfa2-a97b2cc4571c admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:43.583798 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa91960-c5d2-439d-bfa2-a97b2cc4571c admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:43.584080 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa91960-c5d2-439d-bfa2-a97b2cc4571c admin admin] PATH_INFO: `/v3/projects/af0aad12da0442f8803a1543bc2103a0/users/9a4c9090b9694d289de5f5ef768055af/roles/3771341b3be641139e752aab690397a4` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:43.598777 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa91960-c5d2-439d-bfa2-a97b2cc4571c admin admin] RBAC: Authorizing `identity:create_grant(project_id=af0aad12da0442f8803a1543bc2103a0, user_id=9a4c9090b9694d289de5f5ef768055af, role_id=3771341b3be641139e752aab690397a4)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:43.603851 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa91960-c5d2-439d-bfa2-a97b2cc4571c admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:43.677188 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1110/2248] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:41:43 2022] PUT /identity/v3/projects/af0aad12da0442f8803a1543bc2103a0/users/9a4c9090b9694d289de5f5ef768055af/roles/3771341b3be641139e752aab690397a4 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:41:43.683007 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3983b7d-a5d5-41ac-852a-82d1ee025534 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:43.683304 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3983b7d-a5d5-41ac-852a-82d1ee025534 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:43.683592 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3983b7d-a5d5-41ac-852a-82d1ee025534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:43.701467 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-e3983b7d-a5d5-41ac-852a-82d1ee025534 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:41:43.702715 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-e3983b7d-a5d5-41ac-852a-82d1ee025534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:41:43.771770 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1139/2249] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:41:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:41:43.781718 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa91960-c5d2-439d-bfa2-a97b2cc4571c admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:43.783538 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-3aa91960-c5d2-439d-bfa2-a97b2cc4571c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:41:43.843177 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa91960-c5d2-439d-bfa2-a97b2cc4571c admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:43.846824 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed193f94-3c8a-4e2b-bd09-ac750888d837 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:43.847480 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed193f94-3c8a-4e2b-bd09-ac750888d837 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:43.847766 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed193f94-3c8a-4e2b-bd09-ac750888d837 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:43.848168 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed193f94-3c8a-4e2b-bd09-ac750888d837 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:43.858119 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed193f94-3c8a-4e2b-bd09-ac750888d837 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:43.859533 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed193f94-3c8a-4e2b-bd09-ac750888d837 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:43.864560 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:41:43.870352 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1111/2250] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:41:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:41:45.312795 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffafc153-875c-4d7c-8b64-1c3d234ee6eb None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:45.313264 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffafc153-875c-4d7c-8b64-1c3d234ee6eb None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:45.313264 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffafc153-875c-4d7c-8b64-1c3d234ee6eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:45.344876 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-ffafc153-875c-4d7c-8b64-1c3d234ee6eb None None] MFA Rules not processed for user `9a4c9090b9694d289de5f5ef768055af`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:41:45.346120 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-ffafc153-875c-4d7c-8b64-1c3d234ee6eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:41:45.359068 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:41:45.420210 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1140/2251] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:41:45 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:41:45.424670 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed193f94-3c8a-4e2b-bd09-ac750888d837 service neutron] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:45.434307 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed193f94-3c8a-4e2b-bd09-ac750888d837 service neutron] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:45.438719 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b21e7c7-5199-440d-9081-ca9a8b5f4648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:45.441286 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b21e7c7-5199-440d-9081-ca9a8b5f4648 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:45.441286 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b21e7c7-5199-440d-9081-ca9a8b5f4648 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:45.441286 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b21e7c7-5199-440d-9081-ca9a8b5f4648 admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:45.441286 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b21e7c7-5199-440d-9081-ca9a8b5f4648 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:45.442497 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b21e7c7-5199-440d-9081-ca9a8b5f4648 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:45.557699 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1112/2252] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:41:45 2022] POST /identity/v3/projects => generated 373 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:41:45.561529 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffafc153-875c-4d7c-8b64-1c3d234ee6eb None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:45.572730 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffafc153-875c-4d7c-8b64-1c3d234ee6eb None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:45.633975 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5f2526-c6f3-40cd-9a31-d6f30c1b86d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:45.633975 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f2526-c6f3-40cd-9a31-d6f30c1b86d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:45.633975 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f2526-c6f3-40cd-9a31-d6f30c1b86d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:45.633975 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f2526-c6f3-40cd-9a31-d6f30c1b86d2 admin admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:45.634390 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5f2526-c6f3-40cd-9a31-d6f30c1b86d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:45.634390 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5f2526-c6f3-40cd-9a31-d6f30c1b86d2 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:45.711454 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1141/2253] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:41:45 2022] POST /identity/v3/users => generated 293 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:41:45.715403 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b21e7c7-5199-440d-9081-ca9a8b5f4648 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:45.726823 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b21e7c7-5199-440d-9081-ca9a8b5f4648 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:45.736972 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f02986e-6307-43cc-abe0-805e57cd656a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:45.737734 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f02986e-6307-43cc-abe0-805e57cd656a admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:45.739639 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f02986e-6307-43cc-abe0-805e57cd656a admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:45.739639 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f02986e-6307-43cc-abe0-805e57cd656a admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:45.739639 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f02986e-6307-43cc-abe0-805e57cd656a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:45.741442 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f02986e-6307-43cc-abe0-805e57cd656a admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:45.753029 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1113/2254] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:41:45 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:41:45.757739 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5f2526-c6f3-40cd-9a31-d6f30c1b86d2 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:45.768206 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5f2526-c6f3-40cd-9a31-d6f30c1b86d2 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:45.773662 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9963de3b-2a63-45ce-8b8c-a71badc19482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:45.774547 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9963de3b-2a63-45ce-8b8c-a71badc19482 admin admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:45.774834 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9963de3b-2a63-45ce-8b8c-a71badc19482 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:45.775111 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9963de3b-2a63-45ce-8b8c-a71badc19482 admin admin] PATH_INFO: `/v3/projects/d8e5455228c744d8bbef6c90e2357f84/users/7e252a39c47049c2998512109e802263/roles/9baff2b18f2649f7b757ff2e1f63c853` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:45.789958 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9963de3b-2a63-45ce-8b8c-a71badc19482 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8e5455228c744d8bbef6c90e2357f84, user_id=7e252a39c47049c2998512109e802263, role_id=9baff2b18f2649f7b757ff2e1f63c853)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:45.792623 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9963de3b-2a63-45ce-8b8c-a71badc19482 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:45.939681 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1142/2255] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:41:45 2022] PUT /identity/v3/projects/d8e5455228c744d8bbef6c90e2357f84/users/7e252a39c47049c2998512109e802263/roles/9baff2b18f2649f7b757ff2e1f63c853 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:41:45.944367 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f02986e-6307-43cc-abe0-805e57cd656a admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:45.954524 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f02986e-6307-43cc-abe0-805e57cd656a admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:46.012561 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0217673-88e8-4673-8677-5fbb18158468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:46.013325 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0217673-88e8-4673-8677-5fbb18158468 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:46.013622 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0217673-88e8-4673-8677-5fbb18158468 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:46.013906 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0217673-88e8-4673-8677-5fbb18158468 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:46.014638 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0217673-88e8-4673-8677-5fbb18158468 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:46.016528 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0217673-88e8-4673-8677-5fbb18158468 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:46.028035 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1114/2256] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:41:45 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:41:46.032202 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9963de3b-2a63-45ce-8b8c-a71badc19482 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:46.042732 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9963de3b-2a63-45ce-8b8c-a71badc19482 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:46.047832 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d95b8ac-ed96-49c6-84cf-5a904732f21d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:46.048750 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d95b8ac-ed96-49c6-84cf-5a904732f21d admin admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:46.049034 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d95b8ac-ed96-49c6-84cf-5a904732f21d admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:46.049310 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d95b8ac-ed96-49c6-84cf-5a904732f21d admin admin] PATH_INFO: `/v3/projects/d8e5455228c744d8bbef6c90e2357f84/users/7e252a39c47049c2998512109e802263/roles/9baff2b18f2649f7b757ff2e1f63c853` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:46.051263 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d95b8ac-ed96-49c6-84cf-5a904732f21d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8e5455228c744d8bbef6c90e2357f84, user_id=7e252a39c47049c2998512109e802263, role_id=9baff2b18f2649f7b757ff2e1f63c853)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:46.054070 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d95b8ac-ed96-49c6-84cf-5a904732f21d admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:46.064561 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1143/2257] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:41:46 2022] PUT /identity/v3/projects/d8e5455228c744d8bbef6c90e2357f84/users/7e252a39c47049c2998512109e802263/roles/9baff2b18f2649f7b757ff2e1f63c853 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:41:47.309237 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b8e880-d04c-45d0-9f8d-b594c62cfc1f None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:47.310148 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b8e880-d04c-45d0-9f8d-b594c62cfc1f None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:47.310486 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b8e880-d04c-45d0-9f8d-b594c62cfc1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:47.339757 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-f2b8e880-d04c-45d0-9f8d-b594c62cfc1f None None] MFA Rules not processed for user `7e252a39c47049c2998512109e802263`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:41:47.341034 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-f2b8e880-d04c-45d0-9f8d-b594c62cfc1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:41:47.413998 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1115/2258] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:41:47 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:41:47.426044 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d95b8ac-ed96-49c6-84cf-5a904732f21d admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:47.428122 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-8d95b8ac-ed96-49c6-84cf-5a904732f21d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:41:47.510452 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:41:47.513431 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d95b8ac-ed96-49c6-84cf-5a904732f21d admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:47.520028 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bfda29-23da-4dc0-8898-5c58a12471d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d81f597cb44c56a966537089bcce4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:47.520028 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bfda29-23da-4dc0-8898-5c58a12471d0 service cinder] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:47.520028 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bfda29-23da-4dc0-8898-5c58a12471d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:47.520028 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bfda29-23da-4dc0-8898-5c58a12471d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:47.580098 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bfda29-23da-4dc0-8898-5c58a12471d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:47.581592 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bfda29-23da-4dc0-8898-5c58a12471d0 service cinder] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:47.591633 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1144/2259] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:41:47 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:41:49.609047 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8ce516-e659-47d9-8579-942333040cfa None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:49.609522 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8ce516-e659-47d9-8579-942333040cfa None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:49.609611 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8ce516-e659-47d9-8579-942333040cfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:49.627244 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-9b8ce516-e659-47d9-8579-942333040cfa None None] MFA Rules not processed for user `48bbf88b32774d5da9ef93ac75686c34`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:41:49.628484 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-9b8ce516-e659-47d9-8579-942333040cfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:41:49.679078 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:41:49.690849 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1116/2260] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:41:49 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:41:50.374847 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bfda29-23da-4dc0-8898-5c58a12471d0 service cinder] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:41:50.384096 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bfda29-23da-4dc0-8898-5c58a12471d0 service cinder] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:41:50.388010 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6ab828-0ac0-41b0-8866-f4fb1644c274 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d81f597cb44c56a966537089bcce4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:41:50.388737 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ab828-0ac0-41b0-8866-f4fb1644c274 service cinder] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:41:50.389034 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ab828-0ac0-41b0-8866-f4fb1644c274 service cinder] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:41:50.389403 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ab828-0ac0-41b0-8866-f4fb1644c274 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:41:50.399269 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6ab828-0ac0-41b0-8866-f4fb1644c274 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:41:50.400775 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6ab828-0ac0-41b0-8866-f4fb1644c274 service cinder] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:41:50.410603 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1145/2261] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:41:50 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:41:59.942730 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=3 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:42:00.406151 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=32 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:42:15.845748 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8ce516-e659-47d9-8579-942333040cfa None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:42:15.845748 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-9b8ce516-e659-47d9-8579-942333040cfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:42:15.954480 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8ce516-e659-47d9-8579-942333040cfa None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:42:15.960170 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fe4c69-8075-42fb-ae65-e3ee917d07a8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6330294b1c8e4dcd85aad189f007b0aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:42:15.960904 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fe4c69-8075-42fb-ae65-e3ee917d07a8 service placement] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:42:15.961877 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fe4c69-8075-42fb-ae65-e3ee917d07a8 service placement] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:42:15.961877 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fe4c69-8075-42fb-ae65-e3ee917d07a8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:42:15.963831 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-34fe4c69-8075-42fb-ae65-e3ee917d07a8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:42:15.978196 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fe4c69-8075-42fb-ae65-e3ee917d07a8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:42:15.979873 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fe4c69-8075-42fb-ae65-e3ee917d07a8 service placement] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:42:15.992036 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1117/2262] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:42:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:42:18.129295 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6ab828-0ac0-41b0-8866-f4fb1644c274 service cinder] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:42:18.130742 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-ec6ab828-0ac0-41b0-8866-f4fb1644c274 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:42:18.201219 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6ab828-0ac0-41b0-8866-f4fb1644c274 service cinder] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:42:18.206058 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd0a3f2-c039-451f-adc9-76fbb05d9e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:42:18.206969 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd0a3f2-c039-451f-adc9-76fbb05d9e3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:42:18.207249 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd0a3f2-c039-451f-adc9-76fbb05d9e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:42:18.207525 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd0a3f2-c039-451f-adc9-76fbb05d9e3d admin admin] PATH_INFO: `/v3/users/a1a6625692d94e41890449d183379f38` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:42:18.208740 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd0a3f2-c039-451f-adc9-76fbb05d9e3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1a6625692d94e41890449d183379f38)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:42:18.210784 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd0a3f2-c039-451f-adc9-76fbb05d9e3d admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:42:18.469528 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-2dd0a3f2-c039-451f-adc9-76fbb05d9e3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1a6625692d94e41890449d183379f38'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:42:18.470240 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-2dd0a3f2-c039-451f-adc9-76fbb05d9e3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1a6625692d94e41890449d183379f38'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:42:18.470848 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-2dd0a3f2-c039-451f-adc9-76fbb05d9e3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1a6625692d94e41890449d183379f38'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:42:18.478881 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-2dd0a3f2-c039-451f-adc9-76fbb05d9e3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1a6625692d94e41890449d183379f38'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:42:18.493283 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-2dd0a3f2-c039-451f-adc9-76fbb05d9e3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1a6625692d94e41890449d183379f38'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:42:18.602606 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1146/2263] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:42:18 2022] DELETE /identity/v3/users/a1a6625692d94e41890449d183379f38 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:42:18.607126 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f8d0f2-f96a-4620-83a5-bb5c384af19b None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:42:18.607596 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f8d0f2-f96a-4620-83a5-bb5c384af19b None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:42:18.607707 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f8d0f2-f96a-4620-83a5-bb5c384af19b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:42:18.629671 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-92f8d0f2-f96a-4620-83a5-bb5c384af19b None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:42:18.630962 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-92f8d0f2-f96a-4620-83a5-bb5c384af19b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:42:18.700499 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1118/2264] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:42:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:42:18.714607 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd0a3f2-c039-451f-adc9-76fbb05d9e3d admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:42:18.714607 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-2dd0a3f2-c039-451f-adc9-76fbb05d9e3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:42:18.780429 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd0a3f2-c039-451f-adc9-76fbb05d9e3d admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:42:18.785116 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c525477f-fec9-4a71-af0a-7f370b0c6b81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:42:18.785272 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c525477f-fec9-4a71-af0a-7f370b0c6b81 service neutron] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:42:18.785482 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c525477f-fec9-4a71-af0a-7f370b0c6b81 service neutron] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:42:18.785764 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c525477f-fec9-4a71-af0a-7f370b0c6b81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:42:18.796604 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c525477f-fec9-4a71-af0a-7f370b0c6b81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:42:18.798177 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c525477f-fec9-4a71-af0a-7f370b0c6b81 service neutron] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:42:18.809584 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1147/2265] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:42:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:42:19.054098 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f8d0f2-f96a-4620-83a5-bb5c384af19b None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:42:19.056171 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-92f8d0f2-f96a-4620-83a5-bb5c384af19b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:42:19.078085 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f8d0f2-f96a-4620-83a5-bb5c384af19b None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:42:19.083056 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3fcc21-c0c3-48f0-9f00-13fe548cd8fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:42:19.084125 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3fcc21-c0c3-48f0-9f00-13fe548cd8fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:42:19.084443 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3fcc21-c0c3-48f0-9f00-13fe548cd8fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:42:19.084732 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3fcc21-c0c3-48f0-9f00-13fe548cd8fa admin admin] PATH_INFO: `/v3/users/9a4c9090b9694d289de5f5ef768055af` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:42:19.086574 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3fcc21-c0c3-48f0-9f00-13fe548cd8fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a4c9090b9694d289de5f5ef768055af)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:42:19.088533 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3fcc21-c0c3-48f0-9f00-13fe548cd8fa admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:42:19.287503 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c525477f-fec9-4a71-af0a-7f370b0c6b81 service neutron] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:42:19.287503 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-c525477f-fec9-4a71-af0a-7f370b0c6b81 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:42:19.301240 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c525477f-fec9-4a71-af0a-7f370b0c6b81 service neutron] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:42:19.310043 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cbd5a8-2616-4709-8569-7e2030d62401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:42:19.312879 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cbd5a8-2616-4709-8569-7e2030d62401 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:42:19.313647 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cbd5a8-2616-4709-8569-7e2030d62401 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:42:19.314555 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cbd5a8-2616-4709-8569-7e2030d62401 admin admin] PATH_INFO: `/v3/projects/6b5c0b32492048eab9662518d83ad6d6` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:42:19.315762 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cbd5a8-2616-4709-8569-7e2030d62401 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b5c0b32492048eab9662518d83ad6d6)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:42:19.317941 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cbd5a8-2616-4709-8569-7e2030d62401 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:42:19.341631 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-6c3fcc21-c0c3-48f0-9f00-13fe548cd8fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a4c9090b9694d289de5f5ef768055af'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:42:19.342241 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:42:19.342500 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-6c3fcc21-c0c3-48f0-9f00-13fe548cd8fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a4c9090b9694d289de5f5ef768055af'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:42:19.343221 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-6c3fcc21-c0c3-48f0-9f00-13fe548cd8fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a4c9090b9694d289de5f5ef768055af'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:42:19.354724 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-6c3fcc21-c0c3-48f0-9f00-13fe548cd8fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a4c9090b9694d289de5f5ef768055af'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:42:19.366353 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:42:19.373013 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-6c3fcc21-c0c3-48f0-9f00-13fe548cd8fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a4c9090b9694d289de5f5ef768055af'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:42:19.432257 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-d9cbd5a8-2616-4709-8569-7e2030d62401 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b5c0b32492048eab9662518d83ad6d6', 'request_id': 'req-d9cbd5a8-2616-4709-8569-7e2030d62401'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:42:19.443607 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1119/2266] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:42:19 2022] DELETE /identity/v3/users/9a4c9090b9694d289de5f5ef768055af => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:42:19.490038 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-d9cbd5a8-2616-4709-8569-7e2030d62401 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b5c0b32492048eab9662518d83ad6d6', 'request_id': 'req-d9cbd5a8-2616-4709-8569-7e2030d62401'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:42:19.497096 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-d9cbd5a8-2616-4709-8569-7e2030d62401 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b5c0b32492048eab9662518d83ad6d6', 'request_id': 'req-d9cbd5a8-2616-4709-8569-7e2030d62401'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:42:19.511043 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-d9cbd5a8-2616-4709-8569-7e2030d62401 admin admin] The token cache is being invalidate because project 6b5c0b32492048eab9662518d83ad6d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:42:19.511333 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-d9cbd5a8-2616-4709-8569-7e2030d62401 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:42:19.512016 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-d9cbd5a8-2616-4709-8569-7e2030d62401 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:42:19.513797 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1148/2267] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:42:19 2022] DELETE /identity/v3/projects/6b5c0b32492048eab9662518d83ad6d6 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:42:19.519405 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef16ec61-1e92-418b-8fb7-b9a900b47e99 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:42:19.519727 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef16ec61-1e92-418b-8fb7-b9a900b47e99 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:42:19.519988 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef16ec61-1e92-418b-8fb7-b9a900b47e99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:42:19.541726 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-ef16ec61-1e92-418b-8fb7-b9a900b47e99 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:42:19.543359 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-ef16ec61-1e92-418b-8fb7-b9a900b47e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:42:19.617898 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1120/2268] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:42:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:42:19.626043 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62598a6-9946-4b6a-8064-7d394144a9b1 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:42:19.626361 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62598a6-9946-4b6a-8064-7d394144a9b1 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:42:19.626646 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62598a6-9946-4b6a-8064-7d394144a9b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:42:19.647676 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-b62598a6-9946-4b6a-8064-7d394144a9b1 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:42:19.649509 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-b62598a6-9946-4b6a-8064-7d394144a9b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:42:19.658120 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1149/2269] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:42:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:42:19.662665 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef16ec61-1e92-418b-8fb7-b9a900b47e99 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:42:19.672925 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef16ec61-1e92-418b-8fb7-b9a900b47e99 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:42:19.677970 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8491b90f-a97e-4b8d-a03a-a24300203b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:42:19.678633 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8491b90f-a97e-4b8d-a03a-a24300203b09 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:42:19.678921 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8491b90f-a97e-4b8d-a03a-a24300203b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:42:19.679201 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8491b90f-a97e-4b8d-a03a-a24300203b09 admin admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:42:19.679993 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8491b90f-a97e-4b8d-a03a-a24300203b09 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:42:19.680308 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8491b90f-a97e-4b8d-a03a-a24300203b09 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:42:19.681979 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8491b90f-a97e-4b8d-a03a-a24300203b09 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:42:19.693773 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1121/2270] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jun 7 14:42:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:42:19.698742 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62598a6-9946-4b6a-8064-7d394144a9b1 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:42:19.708339 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62598a6-9946-4b6a-8064-7d394144a9b1 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:42:19.714030 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9b49fb-c3de-488f-a07c-731492b0d0e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:42:19.714720 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9b49fb-c3de-488f-a07c-731492b0d0e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:42:19.715036 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9b49fb-c3de-488f-a07c-731492b0d0e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:42:19.715317 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9b49fb-c3de-488f-a07c-731492b0d0e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:42:19.716118 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9b49fb-c3de-488f-a07c-731492b0d0e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:42:19.718110 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9b49fb-c3de-488f-a07c-731492b0d0e8 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:42:19.737516 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8491b90f-a97e-4b8d-a03a-a24300203b09 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:42:19.740232 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-8491b90f-a97e-4b8d-a03a-a24300203b09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:42:19.752951 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8491b90f-a97e-4b8d-a03a-a24300203b09 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:42:19.758356 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a108f4-984b-4ce2-8add-265d54e07ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:42:19.759242 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a108f4-984b-4ce2-8add-265d54e07ee9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:42:19.759524 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a108f4-984b-4ce2-8add-265d54e07ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:42:19.759800 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a108f4-984b-4ce2-8add-265d54e07ee9 admin admin] PATH_INFO: `/v3/projects/af0aad12da0442f8803a1543bc2103a0` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:42:19.761025 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a108f4-984b-4ce2-8add-265d54e07ee9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af0aad12da0442f8803a1543bc2103a0)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:42:19.763390 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a108f4-984b-4ce2-8add-265d54e07ee9 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:42:19.836801 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1150/2271] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:42:19 2022] POST /identity/v3/projects => generated 383 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:42:19.838139 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9b49fb-c3de-488f-a07c-731492b0d0e8 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:42:19.876948 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-23a108f4-984b-4ce2-8add-265d54e07ee9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af0aad12da0442f8803a1543bc2103a0', 'request_id': 'req-23a108f4-984b-4ce2-8add-265d54e07ee9'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:42:19.908709 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9b49fb-c3de-488f-a07c-731492b0d0e8 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:42:19.967566 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b85695a-0b50-464a-ba15-59da5e55275e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:42:19.968218 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-23a108f4-984b-4ce2-8add-265d54e07ee9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af0aad12da0442f8803a1543bc2103a0', 'request_id': 'req-23a108f4-984b-4ce2-8add-265d54e07ee9'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:42:19.968401 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b85695a-0b50-464a-ba15-59da5e55275e admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:42:19.968647 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b85695a-0b50-464a-ba15-59da5e55275e admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:42:19.968932 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b85695a-0b50-464a-ba15-59da5e55275e admin admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:42:19.969658 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b85695a-0b50-464a-ba15-59da5e55275e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:42:19.971730 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b85695a-0b50-464a-ba15-59da5e55275e admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:42:19.974703 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-23a108f4-984b-4ce2-8add-265d54e07ee9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af0aad12da0442f8803a1543bc2103a0', 'request_id': 'req-23a108f4-984b-4ce2-8add-265d54e07ee9'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:42:19.987186 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-23a108f4-984b-4ce2-8add-265d54e07ee9 admin admin] The token cache is being invalidate because project af0aad12da0442f8803a1543bc2103a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:42:19.987478 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-23a108f4-984b-4ce2-8add-265d54e07ee9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:42:19.988171 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-23a108f4-984b-4ce2-8add-265d54e07ee9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:42:19.989800 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1122/2272] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:42:19 2022] DELETE /identity/v3/projects/af0aad12da0442f8803a1543bc2103a0 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:42:19.994365 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a108f4-984b-4ce2-8add-265d54e07ee9 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:42:19.995681 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-23a108f4-984b-4ce2-8add-265d54e07ee9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:42:20.007869 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a108f4-984b-4ce2-8add-265d54e07ee9 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:42:20.013118 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160305be-c5cb-47dd-8663-4beb5144fce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:42:20.014610 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160305be-c5cb-47dd-8663-4beb5144fce7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:42:20.014610 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160305be-c5cb-47dd-8663-4beb5144fce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:42:20.014610 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160305be-c5cb-47dd-8663-4beb5144fce7 admin admin] PATH_INFO: `/v3/users/7e252a39c47049c2998512109e802263` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:42:20.015950 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160305be-c5cb-47dd-8663-4beb5144fce7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e252a39c47049c2998512109e802263)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:42:20.017877 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160305be-c5cb-47dd-8663-4beb5144fce7 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:42:20.040548 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1151/2273] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:42:19 2022] POST /identity/v3/users => generated 292 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:42:20.041758 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b85695a-0b50-464a-ba15-59da5e55275e admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:42:20.043523 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-6b85695a-0b50-464a-ba15-59da5e55275e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:42:20.056264 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b85695a-0b50-464a-ba15-59da5e55275e admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:42:20.062214 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd36d66b-db74-4cb8-8d48-f6cd2a4c3cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:42:20.062889 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd36d66b-db74-4cb8-8d48-f6cd2a4c3cdc admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:42:20.063177 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd36d66b-db74-4cb8-8d48-f6cd2a4c3cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:42:20.063492 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd36d66b-db74-4cb8-8d48-f6cd2a4c3cdc admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:42:20.064219 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd36d66b-db74-4cb8-8d48-f6cd2a4c3cdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:42:20.065863 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd36d66b-db74-4cb8-8d48-f6cd2a4c3cdc admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:42:20.077246 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1152/2274] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:42:20 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 14:42:20.081202 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd36d66b-db74-4cb8-8d48-f6cd2a4c3cdc admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:42:20.086094 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:42:20.091828 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd36d66b-db74-4cb8-8d48-f6cd2a4c3cdc admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:42:20.096471 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7023e21-2076-4247-801a-8b2696e4101c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:42:20.097120 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7023e21-2076-4247-801a-8b2696e4101c admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:42:20.097405 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7023e21-2076-4247-801a-8b2696e4101c admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:42:20.097682 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7023e21-2076-4247-801a-8b2696e4101c admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:42:20.098432 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7023e21-2076-4247-801a-8b2696e4101c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:42:20.100337 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7023e21-2076-4247-801a-8b2696e4101c admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:42:20.110910 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1153/2275] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:42:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 14:42:20.115114 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7023e21-2076-4247-801a-8b2696e4101c admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:42:20.125956 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7023e21-2076-4247-801a-8b2696e4101c admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:42:20.130601 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8983c8a-e017-465d-9640-842fb9459f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:42:20.131472 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8983c8a-e017-465d-9640-842fb9459f20 admin admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:42:20.131760 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8983c8a-e017-465d-9640-842fb9459f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:42:20.132038 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8983c8a-e017-465d-9640-842fb9459f20 admin admin] PATH_INFO: `/v3/projects/634574e706ee4f31bf4446871f11d5e0/users/a605008cb5474f168f572fd0a9e71193/roles/3771341b3be641139e752aab690397a4` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:42:20.148597 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8983c8a-e017-465d-9640-842fb9459f20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=634574e706ee4f31bf4446871f11d5e0, user_id=a605008cb5474f168f572fd0a9e71193, role_id=3771341b3be641139e752aab690397a4)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:42:20.151344 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8983c8a-e017-465d-9640-842fb9459f20 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:42:20.176266 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-160305be-c5cb-47dd-8663-4beb5144fce7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e252a39c47049c2998512109e802263'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:42:20.176939 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-160305be-c5cb-47dd-8663-4beb5144fce7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e252a39c47049c2998512109e802263'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:42:20.177538 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-160305be-c5cb-47dd-8663-4beb5144fce7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e252a39c47049c2998512109e802263'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:42:20.178833 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:42:20.185397 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-160305be-c5cb-47dd-8663-4beb5144fce7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e252a39c47049c2998512109e802263'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:42:20.197567 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-160305be-c5cb-47dd-8663-4beb5144fce7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e252a39c47049c2998512109e802263'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:42:20.260230 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1154/2276] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:42:20 2022] PUT /identity/v3/projects/634574e706ee4f31bf4446871f11d5e0/users/a605008cb5474f168f572fd0a9e71193/roles/3771341b3be641139e752aab690397a4 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:42:20.267577 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1026b2-fed8-4bdd-87f6-0aea44791661 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:42:20.267870 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1026b2-fed8-4bdd-87f6-0aea44791661 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:42:20.268183 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1026b2-fed8-4bdd-87f6-0aea44791661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:42:20.285954 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-bb1026b2-fed8-4bdd-87f6-0aea44791661 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:42:20.292301 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-bb1026b2-fed8-4bdd-87f6-0aea44791661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:42:20.292894 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1123/2277] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:42:19 2022] DELETE /identity/v3/users/7e252a39c47049c2998512109e802263 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:42:20.355086 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1155/2278] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:42:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:42:20.365175 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160305be-c5cb-47dd-8663-4beb5144fce7 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:42:20.366966 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-160305be-c5cb-47dd-8663-4beb5144fce7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:42:20.436788 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160305be-c5cb-47dd-8663-4beb5144fce7 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:42:20.440675 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada9fbff-37b6-414a-9697-0f570f202943 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:42:20.441394 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada9fbff-37b6-414a-9697-0f570f202943 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:42:20.441735 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada9fbff-37b6-414a-9697-0f570f202943 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:42:20.441959 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada9fbff-37b6-414a-9697-0f570f202943 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:42:20.451377 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada9fbff-37b6-414a-9697-0f570f202943 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:42:20.452856 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada9fbff-37b6-414a-9697-0f570f202943 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:42:20.464988 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1124/2279] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:42:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:42:20.565167 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1026b2-fed8-4bdd-87f6-0aea44791661 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:42:20.566768 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-bb1026b2-fed8-4bdd-87f6-0aea44791661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:42:20.581565 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1026b2-fed8-4bdd-87f6-0aea44791661 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:42:20.586849 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad993ca5-8dbc-4429-9e1c-f106946a3eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:42:20.587712 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad993ca5-8dbc-4429-9e1c-f106946a3eb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:42:20.587994 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad993ca5-8dbc-4429-9e1c-f106946a3eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:42:20.588274 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad993ca5-8dbc-4429-9e1c-f106946a3eb8 admin admin] PATH_INFO: `/v3/projects/d8e5455228c744d8bbef6c90e2357f84` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:42:20.589519 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad993ca5-8dbc-4429-9e1c-f106946a3eb8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8e5455228c744d8bbef6c90e2357f84)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:42:20.591595 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad993ca5-8dbc-4429-9e1c-f106946a3eb8 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:42:20.960946 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-ad993ca5-8dbc-4429-9e1c-f106946a3eb8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8e5455228c744d8bbef6c90e2357f84', 'request_id': 'req-ad993ca5-8dbc-4429-9e1c-f106946a3eb8'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:42:21.059844 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-ad993ca5-8dbc-4429-9e1c-f106946a3eb8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8e5455228c744d8bbef6c90e2357f84', 'request_id': 'req-ad993ca5-8dbc-4429-9e1c-f106946a3eb8'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:42:21.066298 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-ad993ca5-8dbc-4429-9e1c-f106946a3eb8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8e5455228c744d8bbef6c90e2357f84', 'request_id': 'req-ad993ca5-8dbc-4429-9e1c-f106946a3eb8'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:42:21.081387 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-ad993ca5-8dbc-4429-9e1c-f106946a3eb8 admin admin] The token cache is being invalidate because project d8e5455228c744d8bbef6c90e2357f84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:42:21.082909 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-ad993ca5-8dbc-4429-9e1c-f106946a3eb8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:42:21.082909 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-ad993ca5-8dbc-4429-9e1c-f106946a3eb8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:42:21.083966 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1156/2280] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:42:20 2022] DELETE /identity/v3/projects/d8e5455228c744d8bbef6c90e2357f84 => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:42:21.089801 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc8bec6-81ae-48b2-927a-5a3a8318e8ff None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:42:21.090516 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc8bec6-81ae-48b2-927a-5a3a8318e8ff None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:42:21.090516 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc8bec6-81ae-48b2-927a-5a3a8318e8ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:42:21.108625 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-9bc8bec6-81ae-48b2-927a-5a3a8318e8ff None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:42:21.109878 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-9bc8bec6-81ae-48b2-927a-5a3a8318e8ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:42:21.172849 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1125/2281] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:42:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:42:21.181653 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098d5db4-617f-4223-8143-4cb9708a8366 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:42:21.182061 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098d5db4-617f-4223-8143-4cb9708a8366 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:42:21.182417 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098d5db4-617f-4223-8143-4cb9708a8366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:42:21.202264 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-098d5db4-617f-4223-8143-4cb9708a8366 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:42:21.203548 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-098d5db4-617f-4223-8143-4cb9708a8366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:42:21.214961 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1157/2282] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:42:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:42:21.217872 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc8bec6-81ae-48b2-927a-5a3a8318e8ff None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:42:21.227552 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc8bec6-81ae-48b2-927a-5a3a8318e8ff None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:42:21.232382 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e726bd0-db05-4496-9cdb-0cc022640dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:42:21.233050 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e726bd0-db05-4496-9cdb-0cc022640dc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:42:21.233337 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e726bd0-db05-4496-9cdb-0cc022640dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:42:21.233617 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e726bd0-db05-4496-9cdb-0cc022640dc3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:42:21.234348 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e726bd0-db05-4496-9cdb-0cc022640dc3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:42:21.234650 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e726bd0-db05-4496-9cdb-0cc022640dc3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:42:21.236306 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e726bd0-db05-4496-9cdb-0cc022640dc3 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:42:21.248150 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1126/2283] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jun 7 14:42:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:42:21.252621 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098d5db4-617f-4223-8143-4cb9708a8366 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:42:21.264258 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098d5db4-617f-4223-8143-4cb9708a8366 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:42:21.272571 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb83c717-1cbd-42a7-aed9-b445c90d0a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:42:21.273283 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb83c717-1cbd-42a7-aed9-b445c90d0a05 admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:42:21.274269 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb83c717-1cbd-42a7-aed9-b445c90d0a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:42:21.274559 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb83c717-1cbd-42a7-aed9-b445c90d0a05 admin admin] PATH_INFO: `/v3/projects` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:42:21.275347 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb83c717-1cbd-42a7-aed9-b445c90d0a05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:42:21.282155 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb83c717-1cbd-42a7-aed9-b445c90d0a05 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:42:21.391757 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1158/2284] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:42:21 2022] POST /identity/v3/projects => generated 377 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:42:21.395608 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e726bd0-db05-4496-9cdb-0cc022640dc3 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:42:21.405849 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e726bd0-db05-4496-9cdb-0cc022640dc3 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:42:21.460819 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:42:21.468487 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337d67b4-64d7-474d-99b4-bf4a9f1b555a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:42:21.469100 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d67b4-64d7-474d-99b4-bf4a9f1b555a admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:42:21.469446 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d67b4-64d7-474d-99b4-bf4a9f1b555a admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:42:21.469737 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d67b4-64d7-474d-99b4-bf4a9f1b555a admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:42:21.470450 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337d67b4-64d7-474d-99b4-bf4a9f1b555a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:42:21.472508 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337d67b4-64d7-474d-99b4-bf4a9f1b555a admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:42:21.585783 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1127/2285] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:42:21 2022] POST /identity/v3/users => generated 289 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:42:21.595691 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb83c717-1cbd-42a7-aed9-b445c90d0a05 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:42:21.600065 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb83c717-1cbd-42a7-aed9-b445c90d0a05 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:42:21.605115 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1960333-f1a9-4bae-a116-ca892472abae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:42:21.605805 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1960333-f1a9-4bae-a116-ca892472abae admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:42:21.606072 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1960333-f1a9-4bae-a116-ca892472abae admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:42:21.606363 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1960333-f1a9-4bae-a116-ca892472abae admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:42:21.607109 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1960333-f1a9-4bae-a116-ca892472abae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:42:21.608815 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1960333-f1a9-4bae-a116-ca892472abae admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:42:21.621720 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1159/2286] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:42:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:42:21.626021 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337d67b4-64d7-474d-99b4-bf4a9f1b555a admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:42:21.636891 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337d67b4-64d7-474d-99b4-bf4a9f1b555a admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:42:21.642966 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e5208a-cfcc-4347-be54-457aace3e708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:42:21.642966 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e5208a-cfcc-4347-be54-457aace3e708 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:42:21.642966 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e5208a-cfcc-4347-be54-457aace3e708 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:42:21.643319 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e5208a-cfcc-4347-be54-457aace3e708 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:42:21.643768 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e5208a-cfcc-4347-be54-457aace3e708 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:42:21.645381 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e5208a-cfcc-4347-be54-457aace3e708 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:42:21.656336 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1128/2287] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:42:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 14:42:21.661199 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1960333-f1a9-4bae-a116-ca892472abae admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:42:21.671302 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1960333-f1a9-4bae-a116-ca892472abae admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:42:21.675991 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281c2f0e-4d24-4623-bd03-34e6bc7b2b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:42:21.680901 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281c2f0e-4d24-4623-bd03-34e6bc7b2b20 admin admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:42:21.680901 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281c2f0e-4d24-4623-bd03-34e6bc7b2b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:42:21.680901 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281c2f0e-4d24-4623-bd03-34e6bc7b2b20 admin admin] PATH_INFO: `/v3/projects/93e877695d0f46339cb4b9b86aac3662/users/0a15f5fa82414bdf8ca3f6723c0a8d2d/roles/3771341b3be641139e752aab690397a4` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:42:21.692712 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:42:21.700873 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281c2f0e-4d24-4623-bd03-34e6bc7b2b20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93e877695d0f46339cb4b9b86aac3662, user_id=0a15f5fa82414bdf8ca3f6723c0a8d2d, role_id=3771341b3be641139e752aab690397a4)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:42:21.703549 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281c2f0e-4d24-4623-bd03-34e6bc7b2b20 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:42:21.803300 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1160/2288] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:42:21 2022] PUT /identity/v3/projects/93e877695d0f46339cb4b9b86aac3662/users/0a15f5fa82414bdf8ca3f6723c0a8d2d/roles/3771341b3be641139e752aab690397a4 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:42:21.809202 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6edffd7-5af6-47fa-bcbb-e310de6dd1c6 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:42:21.809496 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6edffd7-5af6-47fa-bcbb-e310de6dd1c6 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:42:21.809779 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6edffd7-5af6-47fa-bcbb-e310de6dd1c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:42:21.827357 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-b6edffd7-5af6-47fa-bcbb-e310de6dd1c6 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:42:21.828842 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-b6edffd7-5af6-47fa-bcbb-e310de6dd1c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:42:21.890348 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1129/2289] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:42:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:42:21.900398 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281c2f0e-4d24-4623-bd03-34e6bc7b2b20 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:42:21.902208 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-281c2f0e-4d24-4623-bd03-34e6bc7b2b20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:42:21.964592 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281c2f0e-4d24-4623-bd03-34e6bc7b2b20 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:42:21.969530 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032fa53a-8f1a-4724-a779-a79b3e1b04a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:42:21.970205 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032fa53a-8f1a-4724-a779-a79b3e1b04a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:42:21.970675 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032fa53a-8f1a-4724-a779-a79b3e1b04a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:42:21.970964 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032fa53a-8f1a-4724-a779-a79b3e1b04a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:42:21.982760 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032fa53a-8f1a-4724-a779-a79b3e1b04a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:42:21.984868 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032fa53a-8f1a-4724-a779-a79b3e1b04a1 service neutron] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:42:21.996160 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1161/2290] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:42:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:42:22.042868 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ab3fa9-4346-4b04-b8e8-88f7acfb3264 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:42:22.043590 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ab3fa9-4346-4b04-b8e8-88f7acfb3264 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:42:22.043880 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ab3fa9-4346-4b04-b8e8-88f7acfb3264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:42:22.085418 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-d9ab3fa9-4346-4b04-b8e8-88f7acfb3264 None None] MFA Rules not processed for user `a605008cb5474f168f572fd0a9e71193`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:42:22.087263 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-d9ab3fa9-4346-4b04-b8e8-88f7acfb3264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:42:22.162611 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1130/2291] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:42:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:42:22.178292 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032fa53a-8f1a-4724-a779-a79b3e1b04a1 service neutron] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:42:22.181765 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-032fa53a-8f1a-4724-a779-a79b3e1b04a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:42:22.239705 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032fa53a-8f1a-4724-a779-a79b3e1b04a1 service neutron] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:42:22.243275 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb71f1b-ba48-4c3a-b858-5fb14d052b63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d81f597cb44c56a966537089bcce4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:42:22.247626 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb71f1b-ba48-4c3a-b858-5fb14d052b63 service cinder] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:42:22.247626 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb71f1b-ba48-4c3a-b858-5fb14d052b63 service cinder] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:42:22.247626 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb71f1b-ba48-4c3a-b858-5fb14d052b63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:42:22.252838 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb71f1b-ba48-4c3a-b858-5fb14d052b63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:42:22.254286 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb71f1b-ba48-4c3a-b858-5fb14d052b63 service cinder] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:42:22.263290 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1162/2292] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:42:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:42:25.242829 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f057964-30c0-4ad2-a902-e65ae17903bc None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:42:25.243373 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f057964-30c0-4ad2-a902-e65ae17903bc None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:42:25.243791 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f057964-30c0-4ad2-a902-e65ae17903bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:42:25.262934 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-0f057964-30c0-4ad2-a902-e65ae17903bc None None] MFA Rules not processed for user `a605008cb5474f168f572fd0a9e71193`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:42:25.264434 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-0f057964-30c0-4ad2-a902-e65ae17903bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:42:25.272888 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1131/2293] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:42:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:42:28.040534 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced4bdf-b7c6-401f-95a0-94d813d8e0b5 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:42:28.041039 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced4bdf-b7c6-401f-95a0-94d813d8e0b5 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:42:28.041151 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced4bdf-b7c6-401f-95a0-94d813d8e0b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:42:28.071301 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-8ced4bdf-b7c6-401f-95a0-94d813d8e0b5 None None] MFA Rules not processed for user `0a15f5fa82414bdf8ca3f6723c0a8d2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:42:28.072488 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-8ced4bdf-b7c6-401f-95a0-94d813d8e0b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:42:28.146140 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1163/2294] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:42:28 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:42:28.150997 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f057964-30c0-4ad2-a902-e65ae17903bc None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:42:28.161677 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f057964-30c0-4ad2-a902-e65ae17903bc None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:42:28.168079 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbadae98-e140-4727-85fe-f1388ce72cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:42:28.168852 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbadae98-e140-4727-85fe-f1388ce72cb5 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:42:28.169143 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbadae98-e140-4727-85fe-f1388ce72cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:42:28.169424 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbadae98-e140-4727-85fe-f1388ce72cb5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:42:28.170126 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbadae98-e140-4727-85fe-f1388ce72cb5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:42:28.172131 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbadae98-e140-4727-85fe-f1388ce72cb5 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:42:28.270328 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1132/2295] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:42:28 2022] POST /identity/v3/projects => generated 377 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:42:28.271819 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ced4bdf-b7c6-401f-95a0-94d813d8e0b5 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:42:28.283820 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ced4bdf-b7c6-401f-95a0-94d813d8e0b5 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:42:28.303889 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:42:28.343900 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ebab79-b988-45f5-8821-8cdc03d7aa22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:42:28.344527 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ebab79-b988-45f5-8821-8cdc03d7aa22 admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:42:28.344818 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ebab79-b988-45f5-8821-8cdc03d7aa22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:42:28.345094 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ebab79-b988-45f5-8821-8cdc03d7aa22 admin admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:42:28.345814 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ebab79-b988-45f5-8821-8cdc03d7aa22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:42:28.347747 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ebab79-b988-45f5-8821-8cdc03d7aa22 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:42:28.485632 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1164/2296] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:42:28 2022] POST /identity/v3/users => generated 295 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:42:28.489559 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbadae98-e140-4727-85fe-f1388ce72cb5 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:42:28.501073 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbadae98-e140-4727-85fe-f1388ce72cb5 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:42:28.506531 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2662fc-24f6-4ba6-8385-56d4726c138e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:42:28.510580 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2662fc-24f6-4ba6-8385-56d4726c138e admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:42:28.510580 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2662fc-24f6-4ba6-8385-56d4726c138e admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:42:28.510580 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2662fc-24f6-4ba6-8385-56d4726c138e admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:42:28.510580 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2662fc-24f6-4ba6-8385-56d4726c138e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:42:28.511367 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2662fc-24f6-4ba6-8385-56d4726c138e admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:42:28.524812 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1133/2297] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:42:28 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 14:42:28.529223 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ebab79-b988-45f5-8821-8cdc03d7aa22 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:42:28.540460 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ebab79-b988-45f5-8821-8cdc03d7aa22 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:42:28.545646 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf01ae25-15cb-42c5-9e05-ce43005f44b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:42:28.546707 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf01ae25-15cb-42c5-9e05-ce43005f44b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:42:28.547000 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf01ae25-15cb-42c5-9e05-ce43005f44b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:42:28.547282 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf01ae25-15cb-42c5-9e05-ce43005f44b2 admin admin] PATH_INFO: `/v3/projects/8ec9881abe61493b9758792c827b78bb/users/0d3a7b29c42549c1929e3dcf96b09b1e/roles/9baff2b18f2649f7b757ff2e1f63c853` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:42:28.566756 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf01ae25-15cb-42c5-9e05-ce43005f44b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ec9881abe61493b9758792c827b78bb, user_id=0d3a7b29c42549c1929e3dcf96b09b1e, role_id=9baff2b18f2649f7b757ff2e1f63c853)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:42:28.569687 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf01ae25-15cb-42c5-9e05-ce43005f44b2 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:42:28.645111 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1165/2298] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:42:28 2022] PUT /identity/v3/projects/8ec9881abe61493b9758792c827b78bb/users/0d3a7b29c42549c1929e3dcf96b09b1e/roles/9baff2b18f2649f7b757ff2e1f63c853 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:42:28.650047 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2662fc-24f6-4ba6-8385-56d4726c138e admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:42:28.660927 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2662fc-24f6-4ba6-8385-56d4726c138e admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:42:28.683034 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:42:28.720766 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50ed40d-a29f-40fd-a915-1e818d0d5a1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:42:28.721537 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50ed40d-a29f-40fd-a915-1e818d0d5a1b admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:42:28.721822 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50ed40d-a29f-40fd-a915-1e818d0d5a1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:42:28.722130 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50ed40d-a29f-40fd-a915-1e818d0d5a1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:42:28.722857 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50ed40d-a29f-40fd-a915-1e818d0d5a1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:42:28.724483 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50ed40d-a29f-40fd-a915-1e818d0d5a1b admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:42:28.735156 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1134/2299] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:42:28 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:42:28.739417 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf01ae25-15cb-42c5-9e05-ce43005f44b2 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:42:28.758574 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf01ae25-15cb-42c5-9e05-ce43005f44b2 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:42:28.762967 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6405927b-a7ed-4765-8dfa-c6f44b34209c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:42:28.763898 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6405927b-a7ed-4765-8dfa-c6f44b34209c admin admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:42:28.764185 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6405927b-a7ed-4765-8dfa-c6f44b34209c admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:42:28.764524 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6405927b-a7ed-4765-8dfa-c6f44b34209c admin admin] PATH_INFO: `/v3/projects/8ec9881abe61493b9758792c827b78bb/users/0d3a7b29c42549c1929e3dcf96b09b1e/roles/9baff2b18f2649f7b757ff2e1f63c853` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:42:28.766810 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6405927b-a7ed-4765-8dfa-c6f44b34209c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ec9881abe61493b9758792c827b78bb, user_id=0d3a7b29c42549c1929e3dcf96b09b1e, role_id=9baff2b18f2649f7b757ff2e1f63c853)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:42:28.770209 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6405927b-a7ed-4765-8dfa-c6f44b34209c admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:42:28.782251 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1166/2300] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:42:28 2022] PUT /identity/v3/projects/8ec9881abe61493b9758792c827b78bb/users/0d3a7b29c42549c1929e3dcf96b09b1e/roles/9baff2b18f2649f7b757ff2e1f63c853 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:42:34.948081 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda2a60c-615f-4a9d-86a4-d6538d540c6c None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:42:34.949101 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda2a60c-615f-4a9d-86a4-d6538d540c6c None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:42:34.949531 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda2a60c-615f-4a9d-86a4-d6538d540c6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:42:34.988863 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-dda2a60c-615f-4a9d-86a4-d6538d540c6c None None] MFA Rules not processed for user `0d3a7b29c42549c1929e3dcf96b09b1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:42:34.990180 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-dda2a60c-615f-4a9d-86a4-d6538d540c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:42:35.101057 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1135/2301] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:42:34 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:42:35.115458 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6405927b-a7ed-4765-8dfa-c6f44b34209c admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:42:35.117486 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-6405927b-a7ed-4765-8dfa-c6f44b34209c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:42:35.221138 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6405927b-a7ed-4765-8dfa-c6f44b34209c admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:42:35.227162 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f26220e-d1ed-4b97-877a-de6f4bcd0525 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e857db4902e42989d0d040bbc8517c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:42:35.227814 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f26220e-d1ed-4b97-877a-de6f4bcd0525 service nova] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:42:35.228158 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f26220e-d1ed-4b97-877a-de6f4bcd0525 service nova] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:42:35.228492 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f26220e-d1ed-4b97-877a-de6f4bcd0525 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:42:35.274339 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:42:35.286352 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f26220e-d1ed-4b97-877a-de6f4bcd0525 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:42:35.287967 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f26220e-d1ed-4b97-877a-de6f4bcd0525 service nova] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:42:35.296781 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1167/2302] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:42:35 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:42:45.098099 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=52 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:42:45.326915 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=7 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:42:52.136655 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dcf78a-b736-4d48-831d-dc2713d1c5dc None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:42:52.137271 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dcf78a-b736-4d48-831d-dc2713d1c5dc None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:42:52.137388 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dcf78a-b736-4d48-831d-dc2713d1c5dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:42:52.156926 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-13dcf78a-b736-4d48-831d-dc2713d1c5dc None None] MFA Rules not processed for user `34eb9ec1f3ff44bbb03cf934cb2a04de`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:42:52.158134 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-13dcf78a-b736-4d48-831d-dc2713d1c5dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:42:52.223293 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1136/2303] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:42:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:43:02.215843 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=25 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:43:13.613445 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e844cd41-0b62-4b69-8a0b-94999a537f18 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:43:13.616322 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e844cd41-0b62-4b69-8a0b-94999a537f18 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:43:13.616322 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e844cd41-0b62-4b69-8a0b-94999a537f18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:43:13.635206 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-e844cd41-0b62-4b69-8a0b-94999a537f18 None None] MFA Rules not processed for user `0a15f5fa82414bdf8ca3f6723c0a8d2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:43:13.636558 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-e844cd41-0b62-4b69-8a0b-94999a537f18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:43:13.644085 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1168/2304] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:43:13 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:43:23.628702 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=3 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:43:45.420127 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dcf78a-b736-4d48-831d-dc2713d1c5dc None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:43:45.473877 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dcf78a-b736-4d48-831d-dc2713d1c5dc None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:43:45.477504 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64be2d87-cc4c-47a8-8e8e-4792467bcfb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:43:45.478148 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64be2d87-cc4c-47a8-8e8e-4792467bcfb3 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:43:45.478434 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64be2d87-cc4c-47a8-8e8e-4792467bcfb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:43:45.478712 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64be2d87-cc4c-47a8-8e8e-4792467bcfb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:43:45.488363 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64be2d87-cc4c-47a8-8e8e-4792467bcfb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:43:45.489875 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64be2d87-cc4c-47a8-8e8e-4792467bcfb3 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:43:45.500163 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1137/2305] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:43:45 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:43:55.495309 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=26 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:44:11.076301 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40a7430-0484-434b-ab47-39a390b04d42 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:44:11.076854 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40a7430-0484-434b-ab47-39a390b04d42 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:44:11.079628 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40a7430-0484-434b-ab47-39a390b04d42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:44:11.148529 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-e40a7430-0484-434b-ab47-39a390b04d42 None None] MFA Rules not processed for user `ad9d021b171244d796c320aa9f8450a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:44:11.149673 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-e40a7430-0484-434b-ab47-39a390b04d42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:44:11.216047 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1169/2306] 10.222.0.85 () {58 vars in 1009 bytes} [Tue Jun 7 14:44:11 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:44:11.264960 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64be2d87-cc4c-47a8-8e8e-4792467bcfb3 service neutron] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:44:11.266413 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-64be2d87-cc4c-47a8-8e8e-4792467bcfb3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:44:11.374407 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64be2d87-cc4c-47a8-8e8e-4792467bcfb3 service neutron] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:44:11.379749 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cc94db-6036-4c5d-ac31-77d5ddaaab22 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35aa67025f66459da0c0fc3cbfd45430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:44:11.380434 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc94db-6036-4c5d-ac31-77d5ddaaab22 service swift] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:44:11.380727 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc94db-6036-4c5d-ac31-77d5ddaaab22 service swift] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:44:11.381010 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc94db-6036-4c5d-ac31-77d5ddaaab22 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:44:11.389390 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cc94db-6036-4c5d-ac31-77d5ddaaab22 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:44:11.390982 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cc94db-6036-4c5d-ac31-77d5ddaaab22 service swift] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:44:11.399749 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1138/2307] 10.222.0.85 () {60 vars in 1384 bytes} [Tue Jun 7 14:44:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 07 14:44:16.318173 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40a7430-0484-434b-ab47-39a390b04d42 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:44:16.319556 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-e40a7430-0484-434b-ab47-39a390b04d42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:44:16.332623 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40a7430-0484-434b-ab47-39a390b04d42 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:44:16.336495 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229b528e-4306-4cc8-bc59-fe39d580c115 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:44:16.337166 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b528e-4306-4cc8-bc59-fe39d580c115 service neutron] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:44:16.337441 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b528e-4306-4cc8-bc59-fe39d580c115 service neutron] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:44:16.337724 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b528e-4306-4cc8-bc59-fe39d580c115 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:44:16.339583 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-229b528e-4306-4cc8-bc59-fe39d580c115 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:44:16.355855 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229b528e-4306-4cc8-bc59-fe39d580c115 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:44:16.357353 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229b528e-4306-4cc8-bc59-fe39d580c115 service neutron] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:44:16.372273 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1170/2308] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:44:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:44:16.487631 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1915c0-33b7-4068-ba62-0a099514cbd7 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:44:16.488270 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1915c0-33b7-4068-ba62-0a099514cbd7 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:44:16.488815 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1915c0-33b7-4068-ba62-0a099514cbd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:44:16.529369 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-af1915c0-33b7-4068-ba62-0a099514cbd7 None None] MFA Rules not processed for user `a605008cb5474f168f572fd0a9e71193`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:44:16.530705 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-af1915c0-33b7-4068-ba62-0a099514cbd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:44:16.599329 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1139/2309] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:44:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:44:16.663592 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229b528e-4306-4cc8-bc59-fe39d580c115 service neutron] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:44:16.667662 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-229b528e-4306-4cc8-bc59-fe39d580c115 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:44:16.803345 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229b528e-4306-4cc8-bc59-fe39d580c115 service neutron] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:44:16.807694 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c2645b-22fa-464f-b880-35bfc7e7ed67 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51da9867d9147f198b8077e953c4f9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:44:16.808351 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c2645b-22fa-464f-b880-35bfc7e7ed67 service glance] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:44:16.808717 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c2645b-22fa-464f-b880-35bfc7e7ed67 service glance] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:44:16.809097 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c2645b-22fa-464f-b880-35bfc7e7ed67 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:44:16.810952 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-98c2645b-22fa-464f-b880-35bfc7e7ed67 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:44:16.827053 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c2645b-22fa-464f-b880-35bfc7e7ed67 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:44:16.828541 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c2645b-22fa-464f-b880-35bfc7e7ed67 service glance] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:44:16.841172 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1171/2310] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:44:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:44:17.098244 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1915c0-33b7-4068-ba62-0a099514cbd7 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:44:17.111780 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1915c0-33b7-4068-ba62-0a099514cbd7 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:44:17.116245 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd09c5b-f653-4048-b227-5a31b4706295 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:44:17.117018 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd09c5b-f653-4048-b227-5a31b4706295 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:44:17.117383 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd09c5b-f653-4048-b227-5a31b4706295 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:44:17.117778 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd09c5b-f653-4048-b227-5a31b4706295 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:44:17.119937 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-ccd09c5b-f653-4048-b227-5a31b4706295 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:44:17.132441 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd09c5b-f653-4048-b227-5a31b4706295 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:44:17.133970 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd09c5b-f653-4048-b227-5a31b4706295 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:44:17.146757 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1140/2311] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:44:17 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:44:26.835785 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=85 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:44:27.139268 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=72 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:44:29.338959 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c2645b-22fa-464f-b880-35bfc7e7ed67 service glance] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:44:29.340402 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-98c2645b-22fa-464f-b880-35bfc7e7ed67 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:44:29.396712 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c2645b-22fa-464f-b880-35bfc7e7ed67 service glance] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:44:29.400486 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6f6294-fc2d-4b87-9174-3c66c8b461d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d81f597cb44c56a966537089bcce4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:44:29.401138 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6f6294-fc2d-4b87-9174-3c66c8b461d6 service cinder] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:44:29.401426 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6f6294-fc2d-4b87-9174-3c66c8b461d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:44:29.401709 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6f6294-fc2d-4b87-9174-3c66c8b461d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:44:29.403514 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-2d6f6294-fc2d-4b87-9174-3c66c8b461d6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:44:29.461069 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6f6294-fc2d-4b87-9174-3c66c8b461d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:44:29.462460 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6f6294-fc2d-4b87-9174-3c66c8b461d6 service cinder] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:44:29.471418 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1172/2312] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:44:29 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:44:31.634017 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd09c5b-f653-4048-b227-5a31b4706295 service neutron] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:44:31.638110 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd09c5b-f653-4048-b227-5a31b4706295 service neutron] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:44:31.642926 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99488cd1-e5f0-4dd8-a101-e251b523d51a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:44:31.643669 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99488cd1-e5f0-4dd8-a101-e251b523d51a service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:44:31.644424 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99488cd1-e5f0-4dd8-a101-e251b523d51a service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:44:31.644821 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99488cd1-e5f0-4dd8-a101-e251b523d51a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:44:31.647228 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-99488cd1-e5f0-4dd8-a101-e251b523d51a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:44:31.657314 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99488cd1-e5f0-4dd8-a101-e251b523d51a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:44:31.658724 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99488cd1-e5f0-4dd8-a101-e251b523d51a service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:44:31.668064 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1141/2313] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:44:31 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:44:32.742264 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aca727-0dc2-4a77-9c74-4798a31f40ad None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:44:32.743486 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aca727-0dc2-4a77-9c74-4798a31f40ad None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:44:32.743940 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aca727-0dc2-4a77-9c74-4798a31f40ad None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:44:32.746762 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1173/2314] 10.222.0.85 () {54 vars in 921 bytes} [Tue Jun 7 14:44:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:44:32.752783 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e38a49f-d466-44fb-96ee-7cedf86305f1 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:44:32.753233 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e38a49f-d466-44fb-96ee-7cedf86305f1 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:44:32.753533 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e38a49f-d466-44fb-96ee-7cedf86305f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:44:32.804872 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-3e38a49f-d466-44fb-96ee-7cedf86305f1 None None] MFA Rules not processed for user `8e857db4902e42989d0d040bbc8517c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:44:32.806002 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-3e38a49f-d466-44fb-96ee-7cedf86305f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:44:32.817341 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1142/2315] 10.222.0.85 () {58 vars in 1018 bytes} [Tue Jun 7 14:44:32 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:44:32.829071 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aca727-0dc2-4a77-9c74-4798a31f40ad None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:44:32.840975 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aca727-0dc2-4a77-9c74-4798a31f40ad None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:44:32.845113 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099653ef-1bae-4e59-8284-33e468663940 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51da9867d9147f198b8077e953c4f9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:44:32.846095 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099653ef-1bae-4e59-8284-33e468663940 service glance] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:44:32.846555 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099653ef-1bae-4e59-8284-33e468663940 service glance] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:44:32.846999 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099653ef-1bae-4e59-8284-33e468663940 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:44:32.858844 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099653ef-1bae-4e59-8284-33e468663940 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:44:32.860395 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099653ef-1bae-4e59-8284-33e468663940 service glance] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:44:32.871936 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1174/2316] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:44:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:44:33.061697 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cef8b26-4240-4fc3-bf14-c23f7458754d None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:44:33.061994 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cef8b26-4240-4fc3-bf14-c23f7458754d None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:44:33.062280 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cef8b26-4240-4fc3-bf14-c23f7458754d None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:44:33.063656 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1143/2317] 10.222.0.85 () {54 vars in 921 bytes} [Tue Jun 7 14:44:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:44:33.076023 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4709ba1-3238-4ad6-866f-18c1e19de942 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:44:33.076355 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4709ba1-3238-4ad6-866f-18c1e19de942 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:44:33.076677 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4709ba1-3238-4ad6-866f-18c1e19de942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:44:33.095405 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-b4709ba1-3238-4ad6-866f-18c1e19de942 None None] MFA Rules not processed for user `7ac4c85918c045b793a1a53300c989a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:44:33.096660 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-b4709ba1-3238-4ad6-866f-18c1e19de942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:44:33.103516 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1175/2318] 10.222.0.85 () {58 vars in 1018 bytes} [Tue Jun 7 14:44:33 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:44:33.115424 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cef8b26-4240-4fc3-bf14-c23f7458754d None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:44:33.126225 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cef8b26-4240-4fc3-bf14-c23f7458754d None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:44:33.141396 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a933aebd-6fa8-48d2-9e0d-12422f8c9af1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:44:33.142251 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a933aebd-6fa8-48d2-9e0d-12422f8c9af1 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:44:33.142610 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a933aebd-6fa8-48d2-9e0d-12422f8c9af1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:44:33.143195 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a933aebd-6fa8-48d2-9e0d-12422f8c9af1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:44:33.156872 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a933aebd-6fa8-48d2-9e0d-12422f8c9af1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:44:33.158618 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a933aebd-6fa8-48d2-9e0d-12422f8c9af1 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:44:33.168210 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1144/2319] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:44:33 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 14:44:33.264668 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db14e016-4070-460e-8c8d-5c32c28f800e None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:44:33.264967 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db14e016-4070-460e-8c8d-5c32c28f800e None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:44:33.265253 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db14e016-4070-460e-8c8d-5c32c28f800e None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:44:33.266826 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1176/2320] 10.222.0.85 () {54 vars in 921 bytes} [Tue Jun 7 14:44:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:44:33.273622 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdb3c1b-0431-429e-b5aa-d0c737a4203e None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:44:33.273917 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdb3c1b-0431-429e-b5aa-d0c737a4203e None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:44:33.274196 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdb3c1b-0431-429e-b5aa-d0c737a4203e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:44:33.297502 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-0cdb3c1b-0431-429e-b5aa-d0c737a4203e None None] MFA Rules not processed for user `8e857db4902e42989d0d040bbc8517c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:44:33.298642 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-0cdb3c1b-0431-429e-b5aa-d0c737a4203e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:44:33.309445 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1145/2321] 10.222.0.85 () {58 vars in 1018 bytes} [Tue Jun 7 14:44:33 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:44:33.320350 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db14e016-4070-460e-8c8d-5c32c28f800e None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:44:33.331146 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db14e016-4070-460e-8c8d-5c32c28f800e None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:44:33.335682 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778e7292-cbd3-46a1-880c-3aae0b7e495e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:44:33.336304 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778e7292-cbd3-46a1-880c-3aae0b7e495e service neutron] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:44:33.336686 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778e7292-cbd3-46a1-880c-3aae0b7e495e service neutron] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:44:33.337034 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778e7292-cbd3-46a1-880c-3aae0b7e495e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:44:33.347637 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778e7292-cbd3-46a1-880c-3aae0b7e495e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:44:33.351140 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778e7292-cbd3-46a1-880c-3aae0b7e495e service neutron] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:44:33.367953 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1177/2322] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:44:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:44:33.434574 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdb3c1b-0431-429e-b5aa-d0c737a4203e None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:44:33.435947 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-0cdb3c1b-0431-429e-b5aa-d0c737a4203e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:44:33.508804 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdb3c1b-0431-429e-b5aa-d0c737a4203e None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:44:33.513870 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46981468-c167-49a1-b19d-bbed49c974d9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6330294b1c8e4dcd85aad189f007b0aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:44:33.514537 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46981468-c167-49a1-b19d-bbed49c974d9 service placement] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:44:33.514803 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46981468-c167-49a1-b19d-bbed49c974d9 service placement] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:44:33.515077 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46981468-c167-49a1-b19d-bbed49c974d9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:44:33.516892 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-46981468-c167-49a1-b19d-bbed49c974d9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:44:33.529124 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46981468-c167-49a1-b19d-bbed49c974d9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:44:33.530779 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46981468-c167-49a1-b19d-bbed49c974d9 service placement] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:44:33.540803 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1146/2323] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:44:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:44:37.467847 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d26666-0f89-4dd4-8643-5f04598a72d4 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:44:37.468349 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d26666-0f89-4dd4-8643-5f04598a72d4 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:44:37.468462 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d26666-0f89-4dd4-8643-5f04598a72d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:44:37.488136 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-75d26666-0f89-4dd4-8643-5f04598a72d4 None None] MFA Rules not processed for user `ad9d021b171244d796c320aa9f8450a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:44:37.489389 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-75d26666-0f89-4dd4-8643-5f04598a72d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:44:37.497503 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1178/2324] 10.222.0.85 () {58 vars in 1009 bytes} [Tue Jun 7 14:44:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:44:37.509739 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46981468-c167-49a1-b19d-bbed49c974d9 service placement] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:44:37.521867 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46981468-c167-49a1-b19d-bbed49c974d9 service placement] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:44:37.527962 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af53ffd-29bf-4d9a-b776-d9c6af465e1a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35aa67025f66459da0c0fc3cbfd45430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:44:37.528863 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af53ffd-29bf-4d9a-b776-d9c6af465e1a service swift] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:44:37.529154 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af53ffd-29bf-4d9a-b776-d9c6af465e1a service swift] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:44:37.529727 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af53ffd-29bf-4d9a-b776-d9c6af465e1a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:44:37.538935 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af53ffd-29bf-4d9a-b776-d9c6af465e1a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:44:37.540573 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af53ffd-29bf-4d9a-b776-d9c6af465e1a service swift] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:44:37.548759 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1147/2325] 10.222.0.85 () {60 vars in 1384 bytes} [Tue Jun 7 14:44:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 14:44:47.482674 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=66 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:44:47.545459 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=62 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:45:04.587069 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d26666-0f89-4dd4-8643-5f04598a72d4 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:45:04.600116 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d26666-0f89-4dd4-8643-5f04598a72d4 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:45:04.606630 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f1b712-d3dc-40fa-9425-86e1b0696b47 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e857db4902e42989d0d040bbc8517c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:45:04.607280 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f1b712-d3dc-40fa-9425-86e1b0696b47 service nova] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:45:04.607565 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f1b712-d3dc-40fa-9425-86e1b0696b47 service nova] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:45:04.607867 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f1b712-d3dc-40fa-9425-86e1b0696b47 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:45:04.609685 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-35f1b712-d3dc-40fa-9425-86e1b0696b47 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:45:04.668164 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f1b712-d3dc-40fa-9425-86e1b0696b47 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:45:04.669817 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f1b712-d3dc-40fa-9425-86e1b0696b47 service nova] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:45:04.679085 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1179/2326] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:45:04 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:45:13.368970 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af53ffd-29bf-4d9a-b776-d9c6af465e1a service swift] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:45:13.381742 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af53ffd-29bf-4d9a-b776-d9c6af465e1a service swift] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:45:13.388152 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecfabfe-5a7a-4939-9375-16cce87c43b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e857db4902e42989d0d040bbc8517c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:45:13.388855 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecfabfe-5a7a-4939-9375-16cce87c43b2 service nova] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:45:13.389142 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecfabfe-5a7a-4939-9375-16cce87c43b2 service nova] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:45:13.389425 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecfabfe-5a7a-4939-9375-16cce87c43b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:45:13.391385 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-2ecfabfe-5a7a-4939-9375-16cce87c43b2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:45:13.409330 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecfabfe-5a7a-4939-9375-16cce87c43b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:45:13.411484 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecfabfe-5a7a-4939-9375-16cce87c43b2 service nova] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:45:13.422852 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1148/2327] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:45:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:45:14.674580 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=28 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:45:15.852894 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619e6f6a-ca28-4aaa-a234-0023841daa9f None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:45:15.853508 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619e6f6a-ca28-4aaa-a234-0023841daa9f None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:45:15.853508 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619e6f6a-ca28-4aaa-a234-0023841daa9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:45:15.874001 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-619e6f6a-ca28-4aaa-a234-0023841daa9f None None] MFA Rules not processed for user `48bbf88b32774d5da9ef93ac75686c34`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:45:15.875284 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-619e6f6a-ca28-4aaa-a234-0023841daa9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:45:15.883206 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1180/2328] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:45:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:45:19.372888 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22bdfd5-c5a4-4398-9371-13c9a974652b None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:45:19.373385 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22bdfd5-c5a4-4398-9371-13c9a974652b None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:45:19.373504 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22bdfd5-c5a4-4398-9371-13c9a974652b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:45:19.392685 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-d22bdfd5-c5a4-4398-9371-13c9a974652b None None] MFA Rules not processed for user `ad9d021b171244d796c320aa9f8450a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:45:19.394047 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-d22bdfd5-c5a4-4398-9371-13c9a974652b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:45:19.402106 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1149/2329] 10.222.0.85 () {58 vars in 967 bytes} [Tue Jun 7 14:45:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:45:19.424848 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619e6f6a-ca28-4aaa-a234-0023841daa9f None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:45:19.438633 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619e6f6a-ca28-4aaa-a234-0023841daa9f None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:45:19.457865 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55f71ab-fdc7-4949-9d4a-552a75d6d547 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35aa67025f66459da0c0fc3cbfd45430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:45:19.457865 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55f71ab-fdc7-4949-9d4a-552a75d6d547 service swift] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:45:19.457865 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55f71ab-fdc7-4949-9d4a-552a75d6d547 service swift] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:45:19.457865 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55f71ab-fdc7-4949-9d4a-552a75d6d547 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:45:19.458458 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55f71ab-fdc7-4949-9d4a-552a75d6d547 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:45:19.458458 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55f71ab-fdc7-4949-9d4a-552a75d6d547 service swift] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:45:19.465779 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1181/2330] 10.222.0.85 () {60 vars in 1384 bytes} [Tue Jun 7 14:45:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 14:45:26.016996 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22bdfd5-c5a4-4398-9371-13c9a974652b None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:45:26.030102 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22bdfd5-c5a4-4398-9371-13c9a974652b None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:45:26.035927 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a979ff-cb8a-43f7-8edd-e2d3e97ed7e8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e857db4902e42989d0d040bbc8517c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:45:26.036652 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a979ff-cb8a-43f7-8edd-e2d3e97ed7e8 service nova] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:45:26.036942 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a979ff-cb8a-43f7-8edd-e2d3e97ed7e8 service nova] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:45:26.037221 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a979ff-cb8a-43f7-8edd-e2d3e97ed7e8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:45:26.039349 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-48a979ff-cb8a-43f7-8edd-e2d3e97ed7e8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:45:26.052393 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a979ff-cb8a-43f7-8edd-e2d3e97ed7e8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:45:26.053990 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a979ff-cb8a-43f7-8edd-e2d3e97ed7e8 service nova] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:45:26.067444 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1150/2331] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:45:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 14:45:29.461019 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=9 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:45:36.061913 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=15 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:45:46.158649 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55f71ab-fdc7-4949-9d4a-552a75d6d547 service swift] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:45:46.160057 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-d55f71ab-fdc7-4949-9d4a-552a75d6d547 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:45:46.175739 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55f71ab-fdc7-4949-9d4a-552a75d6d547 service swift] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:45:46.181687 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21a155d-3eb3-40df-a86a-a2b52b06a5dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e857db4902e42989d0d040bbc8517c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:45:46.182327 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21a155d-3eb3-40df-a86a-a2b52b06a5dc service nova] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:45:46.182611 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21a155d-3eb3-40df-a86a-a2b52b06a5dc service nova] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:45:46.182918 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21a155d-3eb3-40df-a86a-a2b52b06a5dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:45:46.184820 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-e21a155d-3eb3-40df-a86a-a2b52b06a5dc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:45:46.196101 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21a155d-3eb3-40df-a86a-a2b52b06a5dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:45:46.197720 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21a155d-3eb3-40df-a86a-a2b52b06a5dc service nova] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:45:46.207379 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1182/2332] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:45:46 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 14:45:56.203325 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=6 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:45:58.062219 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a979ff-cb8a-43f7-8edd-e2d3e97ed7e8 service nova] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:45:58.064172 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-48a979ff-cb8a-43f7-8edd-e2d3e97ed7e8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:45:58.083875 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a979ff-cb8a-43f7-8edd-e2d3e97ed7e8 service nova] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:45:58.089653 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a37441-0ceb-464e-8752-b15f3dcffb29 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6330294b1c8e4dcd85aad189f007b0aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:45:58.091118 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a37441-0ceb-464e-8752-b15f3dcffb29 service placement] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:45:58.091536 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a37441-0ceb-464e-8752-b15f3dcffb29 service placement] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:45:58.092730 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a37441-0ceb-464e-8752-b15f3dcffb29 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:45:58.096206 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-d7a37441-0ceb-464e-8752-b15f3dcffb29 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:45:58.117014 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a37441-0ceb-464e-8752-b15f3dcffb29 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:45:58.118957 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a37441-0ceb-464e-8752-b15f3dcffb29 service placement] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:45:58.131211 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1151/2333] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:45:58 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:45:58.231161 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21a155d-3eb3-40df-a86a-a2b52b06a5dc service nova] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:45:58.233316 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-e21a155d-3eb3-40df-a86a-a2b52b06a5dc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:45:58.308127 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21a155d-3eb3-40df-a86a-a2b52b06a5dc service nova] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:45:58.317753 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1578e533-4540-46c0-8387-ef345ec36c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:45:58.320461 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1578e533-4540-46c0-8387-ef345ec36c51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:45:58.320461 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1578e533-4540-46c0-8387-ef345ec36c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:45:58.320461 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1578e533-4540-46c0-8387-ef345ec36c51 admin admin] PATH_INFO: `/v3/users/a605008cb5474f168f572fd0a9e71193` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:45:58.325104 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1578e533-4540-46c0-8387-ef345ec36c51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a605008cb5474f168f572fd0a9e71193)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:45:58.327087 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1578e533-4540-46c0-8387-ef345ec36c51 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:45:58.488626 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-1578e533-4540-46c0-8387-ef345ec36c51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a605008cb5474f168f572fd0a9e71193'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:45:58.489441 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-1578e533-4540-46c0-8387-ef345ec36c51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a605008cb5474f168f572fd0a9e71193'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:45:58.490044 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-1578e533-4540-46c0-8387-ef345ec36c51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a605008cb5474f168f572fd0a9e71193'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:45:58.497176 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-1578e533-4540-46c0-8387-ef345ec36c51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a605008cb5474f168f572fd0a9e71193'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:45:58.510314 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-1578e533-4540-46c0-8387-ef345ec36c51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a605008cb5474f168f572fd0a9e71193'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:45:58.616522 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1183/2334] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:45:58 2022] DELETE /identity/v3/users/a605008cb5474f168f572fd0a9e71193 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:45:58.886383 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a37441-0ceb-464e-8752-b15f3dcffb29 service placement] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:45:58.898051 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a37441-0ceb-464e-8752-b15f3dcffb29 service placement] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:45:58.959549 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0fc517-0e18-4050-92e1-2a0310e2c9b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:45:58.960451 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0fc517-0e18-4050-92e1-2a0310e2c9b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:45:58.961197 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0fc517-0e18-4050-92e1-2a0310e2c9b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:45:58.961197 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0fc517-0e18-4050-92e1-2a0310e2c9b7 admin admin] PATH_INFO: `/v3/projects/634574e706ee4f31bf4446871f11d5e0` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:45:58.962213 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0fc517-0e18-4050-92e1-2a0310e2c9b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=634574e706ee4f31bf4446871f11d5e0)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:45:58.964195 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0fc517-0e18-4050-92e1-2a0310e2c9b7 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:45:59.056765 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-8e0fc517-0e18-4050-92e1-2a0310e2c9b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '634574e706ee4f31bf4446871f11d5e0', 'request_id': 'req-8e0fc517-0e18-4050-92e1-2a0310e2c9b7'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:45:59.118449 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-8e0fc517-0e18-4050-92e1-2a0310e2c9b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '634574e706ee4f31bf4446871f11d5e0', 'request_id': 'req-8e0fc517-0e18-4050-92e1-2a0310e2c9b7'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:45:59.124064 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-8e0fc517-0e18-4050-92e1-2a0310e2c9b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '634574e706ee4f31bf4446871f11d5e0', 'request_id': 'req-8e0fc517-0e18-4050-92e1-2a0310e2c9b7'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:45:59.136975 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-8e0fc517-0e18-4050-92e1-2a0310e2c9b7 admin admin] The token cache is being invalidate because project 634574e706ee4f31bf4446871f11d5e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:45:59.136975 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-8e0fc517-0e18-4050-92e1-2a0310e2c9b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:45:59.136975 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-8e0fc517-0e18-4050-92e1-2a0310e2c9b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:45:59.138285 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1152/2335] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:45:58 2022] DELETE /identity/v3/projects/634574e706ee4f31bf4446871f11d5e0 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:46:03.292114 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd8e25f-7398-4b71-b287-5155f4c827fb None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:46:03.292711 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd8e25f-7398-4b71-b287-5155f4c827fb None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:46:03.292711 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd8e25f-7398-4b71-b287-5155f4c827fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:46:03.312004 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-6cd8e25f-7398-4b71-b287-5155f4c827fb None None] MFA Rules not processed for user `34eb9ec1f3ff44bbb03cf934cb2a04de`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:46:03.313797 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-6cd8e25f-7398-4b71-b287-5155f4c827fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:46:03.368167 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1184/2336] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:46:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:46:03.666731 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0fc517-0e18-4050-92e1-2a0310e2c9b7 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:46:03.668123 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-8e0fc517-0e18-4050-92e1-2a0310e2c9b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:46:03.725743 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0fc517-0e18-4050-92e1-2a0310e2c9b7 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:46:03.729324 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa96aff-7008-4050-98d6-e02d08e8bbc3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:46:03.729965 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa96aff-7008-4050-98d6-e02d08e8bbc3 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:46:03.730246 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa96aff-7008-4050-98d6-e02d08e8bbc3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:46:03.730516 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa96aff-7008-4050-98d6-e02d08e8bbc3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:46:03.732346 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-faa96aff-7008-4050-98d6-e02d08e8bbc3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:46:03.741754 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa96aff-7008-4050-98d6-e02d08e8bbc3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:46:03.744095 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa96aff-7008-4050-98d6-e02d08e8bbc3 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:46:03.751182 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1153/2337] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:46:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:46:13.361461 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=78,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:46:13.747627 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=83,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:46:19.265643 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd8e25f-7398-4b71-b287-5155f4c827fb None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:46:19.267145 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-6cd8e25f-7398-4b71-b287-5155f4c827fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:46:19.344832 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd8e25f-7398-4b71-b287-5155f4c827fb None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:46:19.350412 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636f9593-9229-467a-9c2f-10f90f48d99a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e857db4902e42989d0d040bbc8517c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:46:19.351079 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636f9593-9229-467a-9c2f-10f90f48d99a service nova] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:46:19.351367 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636f9593-9229-467a-9c2f-10f90f48d99a service nova] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:46:19.351643 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636f9593-9229-467a-9c2f-10f90f48d99a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:46:19.353509 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-636f9593-9229-467a-9c2f-10f90f48d99a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:46:19.366561 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636f9593-9229-467a-9c2f-10f90f48d99a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:46:19.368200 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636f9593-9229-467a-9c2f-10f90f48d99a service nova] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:46:19.379392 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1185/2338] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:46:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:46:29.374911 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=34 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:46:30.761565 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa96aff-7008-4050-98d6-e02d08e8bbc3 service neutron] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:46:30.763612 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-faa96aff-7008-4050-98d6-e02d08e8bbc3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:46:30.779750 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa96aff-7008-4050-98d6-e02d08e8bbc3 service neutron] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:46:30.785415 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8566ea4d-7069-4fba-96dd-97615aed65fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e857db4902e42989d0d040bbc8517c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:46:30.786065 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8566ea4d-7069-4fba-96dd-97615aed65fa service nova] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:46:30.786348 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8566ea4d-7069-4fba-96dd-97615aed65fa service nova] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:46:30.786623 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8566ea4d-7069-4fba-96dd-97615aed65fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:46:30.788695 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-8566ea4d-7069-4fba-96dd-97615aed65fa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:46:30.856454 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8566ea4d-7069-4fba-96dd-97615aed65fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:46:30.858074 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8566ea4d-7069-4fba-96dd-97615aed65fa service nova] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:46:30.868188 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1154/2339] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:46:30 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:46:40.863667 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=28 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:46:45.480944 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7919dca1-f1d3-4050-ae75-a3ef4d5f91e7 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:46:45.481532 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7919dca1-f1d3-4050-ae75-a3ef4d5f91e7 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:46:45.481532 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7919dca1-f1d3-4050-ae75-a3ef4d5f91e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:46:45.499667 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-7919dca1-f1d3-4050-ae75-a3ef4d5f91e7 None None] MFA Rules not processed for user `ed3353d3d8a242e1a6b54d31adf97a6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:46:45.500912 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-7919dca1-f1d3-4050-ae75-a3ef4d5f91e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:46:45.508087 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1186/2340] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:46:45 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:46:55.494178 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=3 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:46:57.435995 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8566ea4d-7069-4fba-96dd-97615aed65fa service nova] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:46:57.446926 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8566ea4d-7069-4fba-96dd-97615aed65fa service nova] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:46:57.450702 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf04edf1-bbb3-47b1-b858-6285acd3de0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:46:57.451353 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf04edf1-bbb3-47b1-b858-6285acd3de0d service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:46:57.451764 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf04edf1-bbb3-47b1-b858-6285acd3de0d service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:46:57.452050 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf04edf1-bbb3-47b1-b858-6285acd3de0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:46:57.453885 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-bf04edf1-bbb3-47b1-b858-6285acd3de0d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:46:57.515598 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf04edf1-bbb3-47b1-b858-6285acd3de0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:46:57.517054 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf04edf1-bbb3-47b1-b858-6285acd3de0d service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:46:57.527003 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1155/2341] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:46:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:47:02.697360 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b477abb6-7f2e-4ad3-b680-3bbb687b4cf4 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:02.697833 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b477abb6-7f2e-4ad3-b680-3bbb687b4cf4 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:02.697930 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b477abb6-7f2e-4ad3-b680-3bbb687b4cf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:02.724929 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-b477abb6-7f2e-4ad3-b680-3bbb687b4cf4 None None] MFA Rules not processed for user `4fb4b0cf5f8b4274b787f327793964f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:47:02.726161 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-b477abb6-7f2e-4ad3-b680-3bbb687b4cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:47:02.811560 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1187/2342] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:47:02 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:47:03.016268 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b17585-3430-47b4-a842-356680b306a1 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:03.016783 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b17585-3430-47b4-a842-356680b306a1 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:03.016783 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b17585-3430-47b4-a842-356680b306a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:03.035996 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-e6b17585-3430-47b4-a842-356680b306a1 None None] MFA Rules not processed for user `0a15f5fa82414bdf8ca3f6723c0a8d2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:47:03.037464 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-e6b17585-3430-47b4-a842-356680b306a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:47:03.115108 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1156/2343] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:47:03 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:47:03.437204 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b477abb6-7f2e-4ad3-b680-3bbb687b4cf4 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:47:03.439416 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-b477abb6-7f2e-4ad3-b680-3bbb687b4cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:47:03.453699 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b477abb6-7f2e-4ad3-b680-3bbb687b4cf4 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:47:03.469425 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2d3796-2587-40ac-874e-60549e060787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:47:03.470720 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2d3796-2587-40ac-874e-60549e060787 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:03.471115 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2d3796-2587-40ac-874e-60549e060787 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:03.471596 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2d3796-2587-40ac-874e-60549e060787 admin admin] PATH_INFO: `/v3/users/48bbf88b32774d5da9ef93ac75686c34` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:03.473524 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2d3796-2587-40ac-874e-60549e060787 admin admin] RBAC: Authorizing `identity:delete_user(user_id=48bbf88b32774d5da9ef93ac75686c34)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:47:03.475564 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2d3796-2587-40ac-874e-60549e060787 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:47:03.658036 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-ee2d3796-2587-40ac-874e-60549e060787 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48bbf88b32774d5da9ef93ac75686c34'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:47:03.658754 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-ee2d3796-2587-40ac-874e-60549e060787 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48bbf88b32774d5da9ef93ac75686c34'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:47:03.660040 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-ee2d3796-2587-40ac-874e-60549e060787 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48bbf88b32774d5da9ef93ac75686c34'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:47:03.667296 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-ee2d3796-2587-40ac-874e-60549e060787 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48bbf88b32774d5da9ef93ac75686c34'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:47:03.679741 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-ee2d3796-2587-40ac-874e-60549e060787 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48bbf88b32774d5da9ef93ac75686c34'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:47:03.816992 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1188/2344] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:47:03 2022] DELETE /identity/v3/users/48bbf88b32774d5da9ef93ac75686c34 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:47:04.098350 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b17585-3430-47b4-a842-356680b306a1 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:47:04.111442 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b17585-3430-47b4-a842-356680b306a1 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:47:04.173880 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f35d7eb-a885-4a3f-93e5-480c8383d1ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:47:04.174688 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f35d7eb-a885-4a3f-93e5-480c8383d1ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:04.175551 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f35d7eb-a885-4a3f-93e5-480c8383d1ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:04.175896 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f35d7eb-a885-4a3f-93e5-480c8383d1ac admin admin] PATH_INFO: `/v3/projects/e80aa19d008640df9a1f924a7c03f7f6` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:04.177227 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f35d7eb-a885-4a3f-93e5-480c8383d1ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=e80aa19d008640df9a1f924a7c03f7f6)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:47:04.183632 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f35d7eb-a885-4a3f-93e5-480c8383d1ac admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:47:04.299183 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:47:04.312466 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-4f35d7eb-a885-4a3f-93e5-480c8383d1ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e80aa19d008640df9a1f924a7c03f7f6', 'request_id': 'req-4f35d7eb-a885-4a3f-93e5-480c8383d1ac'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:47:04.386712 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-4f35d7eb-a885-4a3f-93e5-480c8383d1ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e80aa19d008640df9a1f924a7c03f7f6', 'request_id': 'req-4f35d7eb-a885-4a3f-93e5-480c8383d1ac'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:47:04.396250 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-4f35d7eb-a885-4a3f-93e5-480c8383d1ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e80aa19d008640df9a1f924a7c03f7f6', 'request_id': 'req-4f35d7eb-a885-4a3f-93e5-480c8383d1ac'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:47:04.408337 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-4f35d7eb-a885-4a3f-93e5-480c8383d1ac admin admin] The token cache is being invalidate because project e80aa19d008640df9a1f924a7c03f7f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:47:04.408678 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-4f35d7eb-a885-4a3f-93e5-480c8383d1ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:47:04.409910 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-4f35d7eb-a885-4a3f-93e5-480c8383d1ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:47:04.412023 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1157/2345] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:47:04 2022] DELETE /identity/v3/projects/e80aa19d008640df9a1f924a7c03f7f6 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:47:04.416860 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd9f114-2bee-4f67-8697-3f2367c94cdd None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:04.417373 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd9f114-2bee-4f67-8697-3f2367c94cdd None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:04.417661 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd9f114-2bee-4f67-8697-3f2367c94cdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:04.441218 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-3fd9f114-2bee-4f67-8697-3f2367c94cdd None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:47:04.442462 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-3fd9f114-2bee-4f67-8697-3f2367c94cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:47:04.506723 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1189/2346] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:47:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:47:04.515905 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc838ce-a814-42da-b24c-495fca0bee09 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:04.516203 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc838ce-a814-42da-b24c-495fca0bee09 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:04.516528 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc838ce-a814-42da-b24c-495fca0bee09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:04.542177 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-7cc838ce-a814-42da-b24c-495fca0bee09 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:47:04.543482 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-7cc838ce-a814-42da-b24c-495fca0bee09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:47:04.552299 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1158/2347] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:47:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:47:04.556728 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd9f114-2bee-4f67-8697-3f2367c94cdd None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:47:04.567504 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd9f114-2bee-4f67-8697-3f2367c94cdd None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:47:04.572109 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b83b61-1b53-4e00-a869-c08cf5eedb8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:47:04.573118 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b83b61-1b53-4e00-a869-c08cf5eedb8b admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:04.573403 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b83b61-1b53-4e00-a869-c08cf5eedb8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:04.573672 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b83b61-1b53-4e00-a869-c08cf5eedb8b admin admin] PATH_INFO: `/v3/domains` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:04.574399 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b83b61-1b53-4e00-a869-c08cf5eedb8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:47:04.574703 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b83b61-1b53-4e00-a869-c08cf5eedb8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:47:04.576316 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b83b61-1b53-4e00-a869-c08cf5eedb8b admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:47:04.589279 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1190/2348] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jun 7 14:47:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:47:04.593649 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc838ce-a814-42da-b24c-495fca0bee09 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:47:04.605512 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc838ce-a814-42da-b24c-495fca0bee09 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:47:04.612507 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e12096c-b14b-47a6-a3c4-9c0f2320ce68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:47:04.613392 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e12096c-b14b-47a6-a3c4-9c0f2320ce68 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:04.613852 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e12096c-b14b-47a6-a3c4-9c0f2320ce68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:04.614352 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e12096c-b14b-47a6-a3c4-9c0f2320ce68 admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:04.615161 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e12096c-b14b-47a6-a3c4-9c0f2320ce68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:47:04.617638 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e12096c-b14b-47a6-a3c4-9c0f2320ce68 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:47:04.725980 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1159/2349] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:47:04 2022] POST /identity/v3/projects => generated 373 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:47:04.730184 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b83b61-1b53-4e00-a869-c08cf5eedb8b admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:47:04.738520 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:47:04.741498 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b83b61-1b53-4e00-a869-c08cf5eedb8b admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:47:04.811983 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af19baa-7570-41f1-8fca-7f8786e5c6e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:47:04.812711 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af19baa-7570-41f1-8fca-7f8786e5c6e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:04.812992 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af19baa-7570-41f1-8fca-7f8786e5c6e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:04.813272 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af19baa-7570-41f1-8fca-7f8786e5c6e7 admin admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:04.813972 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af19baa-7570-41f1-8fca-7f8786e5c6e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:47:04.815873 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af19baa-7570-41f1-8fca-7f8786e5c6e7 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:47:04.913676 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1191/2350] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:47:04 2022] POST /identity/v3/users => generated 287 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:47:04.917830 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e12096c-b14b-47a6-a3c4-9c0f2320ce68 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:47:04.931228 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e12096c-b14b-47a6-a3c4-9c0f2320ce68 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:47:04.940872 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab3caf3-3727-46f7-b6b1-58e6f5a872bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:47:04.942597 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab3caf3-3727-46f7-b6b1-58e6f5a872bb admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:04.942597 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab3caf3-3727-46f7-b6b1-58e6f5a872bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:04.942597 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab3caf3-3727-46f7-b6b1-58e6f5a872bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:04.943430 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab3caf3-3727-46f7-b6b1-58e6f5a872bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:47:04.945097 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab3caf3-3727-46f7-b6b1-58e6f5a872bb admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:47:04.955760 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1160/2351] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:47:04 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:47:04.960098 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af19baa-7570-41f1-8fca-7f8786e5c6e7 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:47:04.971499 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af19baa-7570-41f1-8fca-7f8786e5c6e7 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:47:04.977643 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5378f516-c86d-47a2-948d-d3cf64088937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:47:04.978378 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5378f516-c86d-47a2-948d-d3cf64088937 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:04.978659 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5378f516-c86d-47a2-948d-d3cf64088937 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:04.978934 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5378f516-c86d-47a2-948d-d3cf64088937 admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:04.979678 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5378f516-c86d-47a2-948d-d3cf64088937 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:47:04.981289 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5378f516-c86d-47a2-948d-d3cf64088937 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:47:04.994600 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1192/2352] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:47:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:47:04.995843 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab3caf3-3727-46f7-b6b1-58e6f5a872bb admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:47:05.007188 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab3caf3-3727-46f7-b6b1-58e6f5a872bb admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:47:05.013166 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02313157-867d-4888-b89f-d84ff76f2e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:47:05.016424 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02313157-867d-4888-b89f-d84ff76f2e60 admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:05.016969 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02313157-867d-4888-b89f-d84ff76f2e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:05.017412 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02313157-867d-4888-b89f-d84ff76f2e60 admin admin] PATH_INFO: `/v3/projects/3d48db3fb13d4a98a2cf40c55cadcf50/users/85ca0eb701f14e75a22a29681c1916a8/roles/3771341b3be641139e752aab690397a4` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:05.037002 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02313157-867d-4888-b89f-d84ff76f2e60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d48db3fb13d4a98a2cf40c55cadcf50, user_id=85ca0eb701f14e75a22a29681c1916a8, role_id=3771341b3be641139e752aab690397a4)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:47:05.040020 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02313157-867d-4888-b89f-d84ff76f2e60 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:47:05.163435 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1161/2353] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:47:04 2022] PUT /identity/v3/projects/3d48db3fb13d4a98a2cf40c55cadcf50/users/85ca0eb701f14e75a22a29681c1916a8/roles/3771341b3be641139e752aab690397a4 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:47:05.176394 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a604bfd9-b4b8-4313-a0d3-014c6e809dc8 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:05.176967 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a604bfd9-b4b8-4313-a0d3-014c6e809dc8 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:05.177272 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a604bfd9-b4b8-4313-a0d3-014c6e809dc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:05.206658 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-a604bfd9-b4b8-4313-a0d3-014c6e809dc8 None None] MFA Rules not processed for user `85ca0eb701f14e75a22a29681c1916a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:47:05.207894 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-a604bfd9-b4b8-4313-a0d3-014c6e809dc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:47:05.293942 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1193/2354] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:47:05 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:47:05.301866 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02313157-867d-4888-b89f-d84ff76f2e60 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:47:05.303733 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-02313157-867d-4888-b89f-d84ff76f2e60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:47:05.396102 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02313157-867d-4888-b89f-d84ff76f2e60 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:47:05.403768 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d40e19-51f3-46e5-a8a8-f828e7a3d9e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e857db4902e42989d0d040bbc8517c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:47:05.404662 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d40e19-51f3-46e5-a8a8-f828e7a3d9e0 service nova] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:05.405055 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d40e19-51f3-46e5-a8a8-f828e7a3d9e0 service nova] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:05.405438 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d40e19-51f3-46e5-a8a8-f828e7a3d9e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:05.435875 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d40e19-51f3-46e5-a8a8-f828e7a3d9e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:47:05.437714 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d40e19-51f3-46e5-a8a8-f828e7a3d9e0 service nova] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:47:05.448928 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1162/2355] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:47:05 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:47:08.923945 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a604bfd9-b4b8-4313-a0d3-014c6e809dc8 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:47:09.042990 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a604bfd9-b4b8-4313-a0d3-014c6e809dc8 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:47:09.047754 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f9a938-c764-4cbd-8318-f7b870e1ca88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:47:09.048732 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f9a938-c764-4cbd-8318-f7b870e1ca88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:09.049020 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f9a938-c764-4cbd-8318-f7b870e1ca88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:09.049301 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f9a938-c764-4cbd-8318-f7b870e1ca88 admin admin] PATH_INFO: `/v3/users/85ca0eb701f14e75a22a29681c1916a8` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:09.050681 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f9a938-c764-4cbd-8318-f7b870e1ca88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85ca0eb701f14e75a22a29681c1916a8)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:47:09.052671 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f9a938-c764-4cbd-8318-f7b870e1ca88 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:47:09.073332 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:47:09.234353 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-97f9a938-c764-4cbd-8318-f7b870e1ca88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85ca0eb701f14e75a22a29681c1916a8'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:47:09.235579 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-97f9a938-c764-4cbd-8318-f7b870e1ca88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85ca0eb701f14e75a22a29681c1916a8'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:47:09.236747 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-97f9a938-c764-4cbd-8318-f7b870e1ca88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85ca0eb701f14e75a22a29681c1916a8'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:47:09.251781 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-97f9a938-c764-4cbd-8318-f7b870e1ca88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85ca0eb701f14e75a22a29681c1916a8'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:47:09.273924 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-97f9a938-c764-4cbd-8318-f7b870e1ca88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85ca0eb701f14e75a22a29681c1916a8'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:47:09.367922 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1194/2356] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:47:08 2022] DELETE /identity/v3/users/85ca0eb701f14e75a22a29681c1916a8 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:47:09.374694 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edcce9e-32d3-43fd-a590-02f736533a21 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:09.375173 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edcce9e-32d3-43fd-a590-02f736533a21 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:09.375318 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edcce9e-32d3-43fd-a590-02f736533a21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:09.393505 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-0edcce9e-32d3-43fd-a590-02f736533a21 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:47:09.395036 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-0edcce9e-32d3-43fd-a590-02f736533a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:47:09.432566 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:47:09.454231 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1163/2357] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:47:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:47:09.463740 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f9a938-c764-4cbd-8318-f7b870e1ca88 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:47:09.465225 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-97f9a938-c764-4cbd-8318-f7b870e1ca88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:47:09.520521 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f9a938-c764-4cbd-8318-f7b870e1ca88 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:47:09.524262 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce328800-2e9f-4940-96b4-3ccc93c91a18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:47:09.524941 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce328800-2e9f-4940-96b4-3ccc93c91a18 service neutron] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:09.525227 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce328800-2e9f-4940-96b4-3ccc93c91a18 service neutron] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:09.525500 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce328800-2e9f-4940-96b4-3ccc93c91a18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:09.534740 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce328800-2e9f-4940-96b4-3ccc93c91a18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:47:09.536158 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce328800-2e9f-4940-96b4-3ccc93c91a18 service neutron] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:47:09.545470 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1195/2358] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:47:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:47:10.014612 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edcce9e-32d3-43fd-a590-02f736533a21 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:47:10.016770 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-0edcce9e-32d3-43fd-a590-02f736533a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:47:10.029315 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edcce9e-32d3-43fd-a590-02f736533a21 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:47:10.034745 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3615e5-749b-4f76-818c-624572b7b72b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:47:10.035739 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3615e5-749b-4f76-818c-624572b7b72b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:10.036230 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3615e5-749b-4f76-818c-624572b7b72b admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:10.036659 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3615e5-749b-4f76-818c-624572b7b72b admin admin] PATH_INFO: `/v3/projects/3d48db3fb13d4a98a2cf40c55cadcf50` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:10.038569 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3615e5-749b-4f76-818c-624572b7b72b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d48db3fb13d4a98a2cf40c55cadcf50)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:47:10.040580 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3615e5-749b-4f76-818c-624572b7b72b admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:47:10.151487 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-4e3615e5-749b-4f76-818c-624572b7b72b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d48db3fb13d4a98a2cf40c55cadcf50', 'request_id': 'req-4e3615e5-749b-4f76-818c-624572b7b72b'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:47:10.241942 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-4e3615e5-749b-4f76-818c-624572b7b72b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d48db3fb13d4a98a2cf40c55cadcf50', 'request_id': 'req-4e3615e5-749b-4f76-818c-624572b7b72b'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:47:10.247759 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-4e3615e5-749b-4f76-818c-624572b7b72b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d48db3fb13d4a98a2cf40c55cadcf50', 'request_id': 'req-4e3615e5-749b-4f76-818c-624572b7b72b'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:47:10.258766 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-4e3615e5-749b-4f76-818c-624572b7b72b admin admin] The token cache is being invalidate because project 3d48db3fb13d4a98a2cf40c55cadcf50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:47:10.259048 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-4e3615e5-749b-4f76-818c-624572b7b72b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:47:10.259675 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-4e3615e5-749b-4f76-818c-624572b7b72b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:47:10.261279 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1164/2359] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:47:10 2022] DELETE /identity/v3/projects/3d48db3fb13d4a98a2cf40c55cadcf50 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:47:10.267286 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f187ca-44dc-4df5-90b4-227c89dc2b26 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:10.267759 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f187ca-44dc-4df5-90b4-227c89dc2b26 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:10.267981 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f187ca-44dc-4df5-90b4-227c89dc2b26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:10.287489 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-d9f187ca-44dc-4df5-90b4-227c89dc2b26 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:47:10.288754 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-d9f187ca-44dc-4df5-90b4-227c89dc2b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:47:10.347784 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1196/2360] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:47:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:47:10.355174 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d0e95-05c0-4173-b8a3-2547da73c158 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:10.355541 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d0e95-05c0-4173-b8a3-2547da73c158 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:10.355822 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d0e95-05c0-4173-b8a3-2547da73c158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:10.373926 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-b24d0e95-05c0-4173-b8a3-2547da73c158 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:47:10.375137 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-b24d0e95-05c0-4173-b8a3-2547da73c158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:47:10.383898 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1165/2361] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:47:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:47:10.388149 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f187ca-44dc-4df5-90b4-227c89dc2b26 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:47:10.398623 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f187ca-44dc-4df5-90b4-227c89dc2b26 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:47:10.403308 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdaa709-9358-42b4-9ac6-8955bad8cbe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:47:10.403979 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdaa709-9358-42b4-9ac6-8955bad8cbe6 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:10.404265 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdaa709-9358-42b4-9ac6-8955bad8cbe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:10.404578 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdaa709-9358-42b4-9ac6-8955bad8cbe6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:10.405320 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdaa709-9358-42b4-9ac6-8955bad8cbe6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:47:10.405619 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdaa709-9358-42b4-9ac6-8955bad8cbe6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:47:10.407219 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdaa709-9358-42b4-9ac6-8955bad8cbe6 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:47:10.420437 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1197/2362] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jun 7 14:47:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:47:10.425270 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24d0e95-05c0-4173-b8a3-2547da73c158 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:47:10.434384 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24d0e95-05c0-4173-b8a3-2547da73c158 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:47:10.439397 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314028ac-1e59-4b43-abee-d0a8d4748a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:47:10.439731 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314028ac-1e59-4b43-abee-d0a8d4748a1d admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:10.440020 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314028ac-1e59-4b43-abee-d0a8d4748a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:10.440298 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314028ac-1e59-4b43-abee-d0a8d4748a1d admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:10.441838 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314028ac-1e59-4b43-abee-d0a8d4748a1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:47:10.443742 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314028ac-1e59-4b43-abee-d0a8d4748a1d admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:47:10.536022 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1166/2363] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:47:10 2022] POST /identity/v3/projects => generated 367 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:47:10.540044 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdaa709-9358-42b4-9ac6-8955bad8cbe6 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:47:10.550968 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdaa709-9358-42b4-9ac6-8955bad8cbe6 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:47:10.569767 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:47:10.606851 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e567d7e-ec25-4d7d-8d71-6e7ad75a6ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:47:10.607489 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e567d7e-ec25-4d7d-8d71-6e7ad75a6ff4 admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:10.607781 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e567d7e-ec25-4d7d-8d71-6e7ad75a6ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:10.608063 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e567d7e-ec25-4d7d-8d71-6e7ad75a6ff4 admin admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:10.608790 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e567d7e-ec25-4d7d-8d71-6e7ad75a6ff4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:47:10.610764 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e567d7e-ec25-4d7d-8d71-6e7ad75a6ff4 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:47:10.729108 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1198/2364] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:47:10 2022] POST /identity/v3/users => generated 284 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:47:10.733301 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314028ac-1e59-4b43-abee-d0a8d4748a1d admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:47:10.744313 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314028ac-1e59-4b43-abee-d0a8d4748a1d admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:47:10.749082 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b00fa1-1cf8-4777-993b-23a7575e6fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:47:10.749738 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b00fa1-1cf8-4777-993b-23a7575e6fcf admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:10.750040 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b00fa1-1cf8-4777-993b-23a7575e6fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:10.750304 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b00fa1-1cf8-4777-993b-23a7575e6fcf admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:10.751058 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b00fa1-1cf8-4777-993b-23a7575e6fcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:47:10.752680 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b00fa1-1cf8-4777-993b-23a7575e6fcf admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:47:10.763219 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1167/2365] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:47:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:47:10.767369 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e567d7e-ec25-4d7d-8d71-6e7ad75a6ff4 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:47:10.777446 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e567d7e-ec25-4d7d-8d71-6e7ad75a6ff4 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:47:10.782072 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b094999-ef65-43fa-afcc-ac3c7b9c9847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:47:10.782711 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b094999-ef65-43fa-afcc-ac3c7b9c9847 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:10.782995 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b094999-ef65-43fa-afcc-ac3c7b9c9847 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:10.783271 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b094999-ef65-43fa-afcc-ac3c7b9c9847 admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:10.784940 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b094999-ef65-43fa-afcc-ac3c7b9c9847 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:47:10.789286 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b094999-ef65-43fa-afcc-ac3c7b9c9847 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:47:10.795899 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1199/2366] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:47:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:47:10.800748 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b00fa1-1cf8-4777-993b-23a7575e6fcf admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:47:10.811249 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b00fa1-1cf8-4777-993b-23a7575e6fcf admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:47:10.815810 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7324efd8-55c1-4d6d-884e-4dddb1cb01e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:47:10.816723 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7324efd8-55c1-4d6d-884e-4dddb1cb01e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:10.817003 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7324efd8-55c1-4d6d-884e-4dddb1cb01e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:10.817274 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7324efd8-55c1-4d6d-884e-4dddb1cb01e0 admin admin] PATH_INFO: `/v3/projects/b4e78468780b4e7ebd88173e202b5163/users/0f410911ac53435f91122cba123ac4af/roles/3771341b3be641139e752aab690397a4` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:10.831964 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7324efd8-55c1-4d6d-884e-4dddb1cb01e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4e78468780b4e7ebd88173e202b5163, user_id=0f410911ac53435f91122cba123ac4af, role_id=3771341b3be641139e752aab690397a4)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:47:10.834675 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7324efd8-55c1-4d6d-884e-4dddb1cb01e0 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:47:10.934914 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1168/2367] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:47:10 2022] PUT /identity/v3/projects/b4e78468780b4e7ebd88173e202b5163/users/0f410911ac53435f91122cba123ac4af/roles/3771341b3be641139e752aab690397a4 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:47:10.946661 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99416c52-1ca0-4a49-8877-2ac1dc8b7024 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:10.946948 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99416c52-1ca0-4a49-8877-2ac1dc8b7024 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:10.947226 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99416c52-1ca0-4a49-8877-2ac1dc8b7024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:10.984140 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-99416c52-1ca0-4a49-8877-2ac1dc8b7024 None None] MFA Rules not processed for user `0f410911ac53435f91122cba123ac4af`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:47:10.985730 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-99416c52-1ca0-4a49-8877-2ac1dc8b7024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:47:11.053312 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1200/2368] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:47:10 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:47:11.057671 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7324efd8-55c1-4d6d-884e-4dddb1cb01e0 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:47:11.125413 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7324efd8-55c1-4d6d-884e-4dddb1cb01e0 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:47:11.130669 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d844facd-6402-4f0c-b25b-c341dc37ae7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:47:11.131059 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d844facd-6402-4f0c-b25b-c341dc37ae7f admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:11.131428 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d844facd-6402-4f0c-b25b-c341dc37ae7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:11.131693 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d844facd-6402-4f0c-b25b-c341dc37ae7f admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:11.132423 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d844facd-6402-4f0c-b25b-c341dc37ae7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:47:11.134297 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d844facd-6402-4f0c-b25b-c341dc37ae7f admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:47:11.138505 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:47:11.227837 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1169/2369] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:47:11 2022] POST /identity/v3/projects => generated 363 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:47:11.231303 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99416c52-1ca0-4a49-8877-2ac1dc8b7024 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:47:11.247755 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99416c52-1ca0-4a49-8877-2ac1dc8b7024 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:47:11.305715 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f8bc53-6960-44e4-abe2-b14d3caf8cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:47:11.306397 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f8bc53-6960-44e4-abe2-b14d3caf8cb7 admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:11.306686 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f8bc53-6960-44e4-abe2-b14d3caf8cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:11.306972 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f8bc53-6960-44e4-abe2-b14d3caf8cb7 admin admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:11.307656 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f8bc53-6960-44e4-abe2-b14d3caf8cb7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:47:11.309611 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f8bc53-6960-44e4-abe2-b14d3caf8cb7 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:47:11.413898 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1201/2370] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:47:11 2022] POST /identity/v3/users => generated 288 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:47:11.417858 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d844facd-6402-4f0c-b25b-c341dc37ae7f admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:47:11.428099 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d844facd-6402-4f0c-b25b-c341dc37ae7f admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:47:11.432775 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28865643-8279-4e20-97b9-4e13e183007b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:47:11.433437 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28865643-8279-4e20-97b9-4e13e183007b admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:11.433722 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28865643-8279-4e20-97b9-4e13e183007b admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:11.434003 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28865643-8279-4e20-97b9-4e13e183007b admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:11.434730 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28865643-8279-4e20-97b9-4e13e183007b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:47:11.436449 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28865643-8279-4e20-97b9-4e13e183007b admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:47:11.447534 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1170/2371] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:47:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:47:11.452017 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f8bc53-6960-44e4-abe2-b14d3caf8cb7 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:47:11.463781 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f8bc53-6960-44e4-abe2-b14d3caf8cb7 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:47:11.468364 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8677c4b-bc2f-4dde-a5fb-a5799a98f9a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:47:11.469347 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8677c4b-bc2f-4dde-a5fb-a5799a98f9a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:11.469752 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8677c4b-bc2f-4dde-a5fb-a5799a98f9a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:11.469878 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8677c4b-bc2f-4dde-a5fb-a5799a98f9a5 admin admin] PATH_INFO: `/v3/projects/2990df044b1444debaf205c5097d9e59/users/cfb4026f179e471386088480cb10c2fd/roles/9baff2b18f2649f7b757ff2e1f63c853` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:11.485876 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8677c4b-bc2f-4dde-a5fb-a5799a98f9a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2990df044b1444debaf205c5097d9e59, user_id=cfb4026f179e471386088480cb10c2fd, role_id=9baff2b18f2649f7b757ff2e1f63c853)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:47:11.488534 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8677c4b-bc2f-4dde-a5fb-a5799a98f9a5 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:47:11.491983 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:47:11.557377 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1202/2372] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:47:11 2022] PUT /identity/v3/projects/2990df044b1444debaf205c5097d9e59/users/cfb4026f179e471386088480cb10c2fd/roles/9baff2b18f2649f7b757ff2e1f63c853 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:47:11.562003 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28865643-8279-4e20-97b9-4e13e183007b admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:47:11.575841 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28865643-8279-4e20-97b9-4e13e183007b admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:47:11.638613 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc0e27e-fe82-4ae2-b53b-d9710db5e818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:47:11.639352 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc0e27e-fe82-4ae2-b53b-d9710db5e818 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:11.639649 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc0e27e-fe82-4ae2-b53b-d9710db5e818 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:11.639933 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc0e27e-fe82-4ae2-b53b-d9710db5e818 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:11.640712 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc0e27e-fe82-4ae2-b53b-d9710db5e818 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:47:11.642371 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc0e27e-fe82-4ae2-b53b-d9710db5e818 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:47:11.657429 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1171/2373] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:47:11 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:47:11.659004 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8677c4b-bc2f-4dde-a5fb-a5799a98f9a5 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:47:11.681708 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8677c4b-bc2f-4dde-a5fb-a5799a98f9a5 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:47:11.686762 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e157e910-268b-4a7f-ba22-5f844631baee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:47:11.688492 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e157e910-268b-4a7f-ba22-5f844631baee admin admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:11.688492 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e157e910-268b-4a7f-ba22-5f844631baee admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:11.688492 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e157e910-268b-4a7f-ba22-5f844631baee admin admin] PATH_INFO: `/v3/projects/2990df044b1444debaf205c5097d9e59/users/cfb4026f179e471386088480cb10c2fd/roles/9baff2b18f2649f7b757ff2e1f63c853` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:11.691228 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e157e910-268b-4a7f-ba22-5f844631baee admin admin] RBAC: Authorizing `identity:create_grant(project_id=2990df044b1444debaf205c5097d9e59, user_id=cfb4026f179e471386088480cb10c2fd, role_id=9baff2b18f2649f7b757ff2e1f63c853)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:47:11.694124 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e157e910-268b-4a7f-ba22-5f844631baee admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:47:11.705120 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1203/2374] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:47:11 2022] PUT /identity/v3/projects/2990df044b1444debaf205c5097d9e59/users/cfb4026f179e471386088480cb10c2fd/roles/9baff2b18f2649f7b757ff2e1f63c853 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:47:11.715248 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d9f1bf-0898-4f83-b1ab-c12558e12bd3 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:11.715539 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d9f1bf-0898-4f83-b1ab-c12558e12bd3 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:11.715822 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d9f1bf-0898-4f83-b1ab-c12558e12bd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:11.743192 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-85d9f1bf-0898-4f83-b1ab-c12558e12bd3 None None] MFA Rules not processed for user `cfb4026f179e471386088480cb10c2fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:47:11.744335 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-85d9f1bf-0898-4f83-b1ab-c12558e12bd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:47:11.813363 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1172/2375] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:47:11 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:47:11.826194 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e157e910-268b-4a7f-ba22-5f844631baee admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:47:11.828573 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-e157e910-268b-4a7f-ba22-5f844631baee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:47:11.896414 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e157e910-268b-4a7f-ba22-5f844631baee admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:47:11.900398 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50535951-ad71-46a8-abf7-0b99e2d90bd3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d81f597cb44c56a966537089bcce4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:47:11.909503 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50535951-ad71-46a8-abf7-0b99e2d90bd3 service cinder] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:11.909503 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50535951-ad71-46a8-abf7-0b99e2d90bd3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:11.909503 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50535951-ad71-46a8-abf7-0b99e2d90bd3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:11.910644 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50535951-ad71-46a8-abf7-0b99e2d90bd3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:47:11.912131 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50535951-ad71-46a8-abf7-0b99e2d90bd3 service cinder] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:47:11.922233 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1204/2376] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:47:11 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:47:12.161694 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d9f1bf-0898-4f83-b1ab-c12558e12bd3 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:47:12.171470 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d9f1bf-0898-4f83-b1ab-c12558e12bd3 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:47:12.175819 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8086536a-e7e0-4403-bba3-5bbc0a97f9f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d81f597cb44c56a966537089bcce4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:47:12.176597 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8086536a-e7e0-4403-bba3-5bbc0a97f9f4 service cinder] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:12.176948 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8086536a-e7e0-4403-bba3-5bbc0a97f9f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:12.177293 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8086536a-e7e0-4403-bba3-5bbc0a97f9f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:12.232918 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:47:12.235408 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8086536a-e7e0-4403-bba3-5bbc0a97f9f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:47:12.236971 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8086536a-e7e0-4403-bba3-5bbc0a97f9f4 service cinder] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:47:12.246680 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1173/2377] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:47:12 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 14:47:16.437111 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50535951-ad71-46a8-abf7-0b99e2d90bd3 service cinder] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:47:16.438536 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-50535951-ad71-46a8-abf7-0b99e2d90bd3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:47:16.515411 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50535951-ad71-46a8-abf7-0b99e2d90bd3 service cinder] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:47:16.524759 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c090b3-a3f2-4fd8-a211-400014dfebc9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6330294b1c8e4dcd85aad189f007b0aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:47:16.525360 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c090b3-a3f2-4fd8-a211-400014dfebc9 service placement] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:16.525644 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c090b3-a3f2-4fd8-a211-400014dfebc9 service placement] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:16.525921 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c090b3-a3f2-4fd8-a211-400014dfebc9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:16.527681 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-b8c090b3-a3f2-4fd8-a211-400014dfebc9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:47:16.539452 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c090b3-a3f2-4fd8-a211-400014dfebc9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:47:16.541125 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c090b3-a3f2-4fd8-a211-400014dfebc9 service placement] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:47:16.551539 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1205/2378] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:47:16 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:47:18.709772 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8086536a-e7e0-4403-bba3-5bbc0a97f9f4 service cinder] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:47:18.777178 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8086536a-e7e0-4403-bba3-5bbc0a97f9f4 service cinder] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:47:18.781952 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230f33a9-cf5f-4d43-bbf6-36a284ee5ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:47:18.782850 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230f33a9-cf5f-4d43-bbf6-36a284ee5ae4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:18.783115 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230f33a9-cf5f-4d43-bbf6-36a284ee5ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:18.783399 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230f33a9-cf5f-4d43-bbf6-36a284ee5ae4 admin admin] PATH_INFO: `/v3/users/0f410911ac53435f91122cba123ac4af` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:18.785233 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230f33a9-cf5f-4d43-bbf6-36a284ee5ae4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f410911ac53435f91122cba123ac4af)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:47:18.787400 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230f33a9-cf5f-4d43-bbf6-36a284ee5ae4 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:47:19.001189 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-230f33a9-cf5f-4d43-bbf6-36a284ee5ae4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f410911ac53435f91122cba123ac4af'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:47:19.001898 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-230f33a9-cf5f-4d43-bbf6-36a284ee5ae4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f410911ac53435f91122cba123ac4af'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:47:19.002570 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-230f33a9-cf5f-4d43-bbf6-36a284ee5ae4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f410911ac53435f91122cba123ac4af'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:47:19.010048 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-230f33a9-cf5f-4d43-bbf6-36a284ee5ae4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f410911ac53435f91122cba123ac4af'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:47:19.023660 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-230f33a9-cf5f-4d43-bbf6-36a284ee5ae4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f410911ac53435f91122cba123ac4af'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:47:19.106265 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1174/2379] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:47:18 2022] DELETE /identity/v3/users/0f410911ac53435f91122cba123ac4af => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:47:19.111366 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14fe8fc-a709-4011-9cc4-8227e82269e1 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:19.111886 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14fe8fc-a709-4011-9cc4-8227e82269e1 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:19.112024 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14fe8fc-a709-4011-9cc4-8227e82269e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:19.131178 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-d14fe8fc-a709-4011-9cc4-8227e82269e1 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:47:19.132413 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-d14fe8fc-a709-4011-9cc4-8227e82269e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:47:19.196878 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1206/2380] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:47:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:47:19.206503 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230f33a9-cf5f-4d43-bbf6-36a284ee5ae4 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:47:19.208743 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-230f33a9-cf5f-4d43-bbf6-36a284ee5ae4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:47:19.270692 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230f33a9-cf5f-4d43-bbf6-36a284ee5ae4 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:47:19.274770 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e470724-f81a-4dc8-aefe-cf0d93fe0417 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:47:19.282301 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e470724-f81a-4dc8-aefe-cf0d93fe0417 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:19.282301 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e470724-f81a-4dc8-aefe-cf0d93fe0417 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:19.282301 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e470724-f81a-4dc8-aefe-cf0d93fe0417 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:19.286293 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e470724-f81a-4dc8-aefe-cf0d93fe0417 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:47:19.287793 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e470724-f81a-4dc8-aefe-cf0d93fe0417 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:47:19.297841 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1175/2381] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:47:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:47:19.751536 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14fe8fc-a709-4011-9cc4-8227e82269e1 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:47:19.753343 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-d14fe8fc-a709-4011-9cc4-8227e82269e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:47:19.781831 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14fe8fc-a709-4011-9cc4-8227e82269e1 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:47:19.789544 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35764cc5-2088-46ff-87a0-6b0ad4e26b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:47:19.790529 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35764cc5-2088-46ff-87a0-6b0ad4e26b10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:19.791085 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35764cc5-2088-46ff-87a0-6b0ad4e26b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:19.791453 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35764cc5-2088-46ff-87a0-6b0ad4e26b10 admin admin] PATH_INFO: `/v3/projects/b4e78468780b4e7ebd88173e202b5163` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:19.793067 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35764cc5-2088-46ff-87a0-6b0ad4e26b10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4e78468780b4e7ebd88173e202b5163)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:47:19.795584 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35764cc5-2088-46ff-87a0-6b0ad4e26b10 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:47:19.808656 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:47:19.950617 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-35764cc5-2088-46ff-87a0-6b0ad4e26b10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4e78468780b4e7ebd88173e202b5163', 'request_id': 'req-35764cc5-2088-46ff-87a0-6b0ad4e26b10'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:47:20.024890 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-35764cc5-2088-46ff-87a0-6b0ad4e26b10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4e78468780b4e7ebd88173e202b5163', 'request_id': 'req-35764cc5-2088-46ff-87a0-6b0ad4e26b10'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:47:20.031432 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-35764cc5-2088-46ff-87a0-6b0ad4e26b10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4e78468780b4e7ebd88173e202b5163', 'request_id': 'req-35764cc5-2088-46ff-87a0-6b0ad4e26b10'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:47:20.044176 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-35764cc5-2088-46ff-87a0-6b0ad4e26b10 admin admin] The token cache is being invalidate because project b4e78468780b4e7ebd88173e202b5163 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:47:20.045166 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-35764cc5-2088-46ff-87a0-6b0ad4e26b10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:47:20.045844 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-35764cc5-2088-46ff-87a0-6b0ad4e26b10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:47:20.047570 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1207/2382] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:47:19 2022] DELETE /identity/v3/projects/b4e78468780b4e7ebd88173e202b5163 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:47:20.052650 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e470724-f81a-4dc8-aefe-cf0d93fe0417 service neutron] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:47:20.062856 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e470724-f81a-4dc8-aefe-cf0d93fe0417 service neutron] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:47:20.078241 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:47:20.132273 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf23ed9-fba7-4025-8898-d4e21192a94c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:47:20.133189 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf23ed9-fba7-4025-8898-d4e21192a94c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:20.133481 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf23ed9-fba7-4025-8898-d4e21192a94c admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:20.133754 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf23ed9-fba7-4025-8898-d4e21192a94c admin admin] PATH_INFO: `/v3/users/cfb4026f179e471386088480cb10c2fd` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:20.134968 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf23ed9-fba7-4025-8898-d4e21192a94c admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfb4026f179e471386088480cb10c2fd)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:47:20.137255 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf23ed9-fba7-4025-8898-d4e21192a94c admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:47:20.311858 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-daf23ed9-fba7-4025-8898-d4e21192a94c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfb4026f179e471386088480cb10c2fd'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:47:20.312572 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-daf23ed9-fba7-4025-8898-d4e21192a94c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfb4026f179e471386088480cb10c2fd'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:47:20.313199 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-daf23ed9-fba7-4025-8898-d4e21192a94c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfb4026f179e471386088480cb10c2fd'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:47:20.322889 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-daf23ed9-fba7-4025-8898-d4e21192a94c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfb4026f179e471386088480cb10c2fd'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:47:20.342527 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-daf23ed9-fba7-4025-8898-d4e21192a94c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfb4026f179e471386088480cb10c2fd'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:47:20.415873 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1176/2383] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:47:20 2022] DELETE /identity/v3/users/cfb4026f179e471386088480cb10c2fd => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:47:20.832785 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35764cc5-2088-46ff-87a0-6b0ad4e26b10 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:47:20.834320 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-35764cc5-2088-46ff-87a0-6b0ad4e26b10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:47:20.849692 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35764cc5-2088-46ff-87a0-6b0ad4e26b10 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:47:20.910436 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9956cdc4-35b3-43ee-a856-0f991a22123b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:47:20.911847 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9956cdc4-35b3-43ee-a856-0f991a22123b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:20.912338 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9956cdc4-35b3-43ee-a856-0f991a22123b admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:20.912681 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9956cdc4-35b3-43ee-a856-0f991a22123b admin admin] PATH_INFO: `/v3/projects/2990df044b1444debaf205c5097d9e59` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:20.913964 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9956cdc4-35b3-43ee-a856-0f991a22123b admin admin] RBAC: Authorizing `identity:delete_project(project_id=2990df044b1444debaf205c5097d9e59)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:47:20.915942 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9956cdc4-35b3-43ee-a856-0f991a22123b admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:47:21.066566 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-9956cdc4-35b3-43ee-a856-0f991a22123b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2990df044b1444debaf205c5097d9e59', 'request_id': 'req-9956cdc4-35b3-43ee-a856-0f991a22123b'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:47:21.140551 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-9956cdc4-35b3-43ee-a856-0f991a22123b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2990df044b1444debaf205c5097d9e59', 'request_id': 'req-9956cdc4-35b3-43ee-a856-0f991a22123b'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:47:21.146812 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-9956cdc4-35b3-43ee-a856-0f991a22123b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2990df044b1444debaf205c5097d9e59', 'request_id': 'req-9956cdc4-35b3-43ee-a856-0f991a22123b'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:47:21.160985 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-9956cdc4-35b3-43ee-a856-0f991a22123b admin admin] The token cache is being invalidate because project 2990df044b1444debaf205c5097d9e59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:47:21.160985 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-9956cdc4-35b3-43ee-a856-0f991a22123b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:47:21.161768 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-9956cdc4-35b3-43ee-a856-0f991a22123b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:47:21.163365 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1208/2384] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:47:20 2022] DELETE /identity/v3/projects/2990df044b1444debaf205c5097d9e59 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:47:21.168178 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa7455d-74d2-41fc-b139-28539588cc55 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:21.168713 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa7455d-74d2-41fc-b139-28539588cc55 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:21.168829 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa7455d-74d2-41fc-b139-28539588cc55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:21.186996 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-afa7455d-74d2-41fc-b139-28539588cc55 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:47:21.188197 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-afa7455d-74d2-41fc-b139-28539588cc55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:47:21.246210 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1177/2385] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:47:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:47:21.254184 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762b4c6-76bc-4b77-bd6c-8cbf0b27858b None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:21.254477 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762b4c6-76bc-4b77-bd6c-8cbf0b27858b None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:21.254766 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762b4c6-76bc-4b77-bd6c-8cbf0b27858b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:21.273994 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-f762b4c6-76bc-4b77-bd6c-8cbf0b27858b None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:47:21.275214 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-f762b4c6-76bc-4b77-bd6c-8cbf0b27858b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:47:21.283514 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1209/2386] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:47:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:47:21.288504 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa7455d-74d2-41fc-b139-28539588cc55 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:47:21.298228 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa7455d-74d2-41fc-b139-28539588cc55 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:47:21.305287 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd64ad16-54cc-494c-8281-6b1ed5d3eb2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:47:21.305963 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd64ad16-54cc-494c-8281-6b1ed5d3eb2c admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:21.306252 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd64ad16-54cc-494c-8281-6b1ed5d3eb2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:21.306533 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd64ad16-54cc-494c-8281-6b1ed5d3eb2c admin admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:21.307281 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd64ad16-54cc-494c-8281-6b1ed5d3eb2c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:47:21.307605 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd64ad16-54cc-494c-8281-6b1ed5d3eb2c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:47:21.309276 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd64ad16-54cc-494c-8281-6b1ed5d3eb2c admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:47:21.321195 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1178/2387] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jun 7 14:47:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:47:21.325573 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f762b4c6-76bc-4b77-bd6c-8cbf0b27858b None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:47:21.334821 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f762b4c6-76bc-4b77-bd6c-8cbf0b27858b None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:47:21.339543 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3ac60f-c62e-4261-a931-81176d83e6e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:47:21.340189 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3ac60f-c62e-4261-a931-81176d83e6e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:21.340513 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3ac60f-c62e-4261-a931-81176d83e6e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:21.340796 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3ac60f-c62e-4261-a931-81176d83e6e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:21.341498 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3ac60f-c62e-4261-a931-81176d83e6e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:47:21.343382 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3ac60f-c62e-4261-a931-81176d83e6e7 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:47:21.455998 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:47:21.463132 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1210/2388] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:47:21 2022] POST /identity/v3/projects => generated 385 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:47:21.467511 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd64ad16-54cc-494c-8281-6b1ed5d3eb2c admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:47:21.478562 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd64ad16-54cc-494c-8281-6b1ed5d3eb2c admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:47:21.499645 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:47:21.535925 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7402c78-ed42-45a2-9b2c-2f076a717077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:47:21.536651 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7402c78-ed42-45a2-9b2c-2f076a717077 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:21.536913 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7402c78-ed42-45a2-9b2c-2f076a717077 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:21.537186 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7402c78-ed42-45a2-9b2c-2f076a717077 admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:21.537867 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7402c78-ed42-45a2-9b2c-2f076a717077 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:47:21.539823 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7402c78-ed42-45a2-9b2c-2f076a717077 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:47:21.666074 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1179/2389] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:47:21 2022] POST /identity/v3/users => generated 293 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:47:21.670147 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3ac60f-c62e-4261-a931-81176d83e6e7 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:47:21.680517 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3ac60f-c62e-4261-a931-81176d83e6e7 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:47:21.685081 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a5775b-5274-4c4b-be0e-bb42acf91bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:47:21.685740 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a5775b-5274-4c4b-be0e-bb42acf91bc6 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:21.686027 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a5775b-5274-4c4b-be0e-bb42acf91bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:21.686307 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a5775b-5274-4c4b-be0e-bb42acf91bc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:21.687227 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a5775b-5274-4c4b-be0e-bb42acf91bc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:47:21.688833 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a5775b-5274-4c4b-be0e-bb42acf91bc6 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:47:21.699054 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1211/2390] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:47:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:47:21.702970 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7402c78-ed42-45a2-9b2c-2f076a717077 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:47:21.713437 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7402c78-ed42-45a2-9b2c-2f076a717077 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:47:21.717958 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bcd115-7f5e-4188-ab49-c7623472bc61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:47:21.718617 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bcd115-7f5e-4188-ab49-c7623472bc61 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:21.718900 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bcd115-7f5e-4188-ab49-c7623472bc61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:21.719172 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bcd115-7f5e-4188-ab49-c7623472bc61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:21.719898 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bcd115-7f5e-4188-ab49-c7623472bc61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:47:21.721517 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bcd115-7f5e-4188-ab49-c7623472bc61 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:47:21.732691 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1180/2391] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:47:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:47:21.736877 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a5775b-5274-4c4b-be0e-bb42acf91bc6 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:47:21.747136 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a5775b-5274-4c4b-be0e-bb42acf91bc6 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:47:21.751866 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260f0c35-383b-4548-8543-65354dda3ec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:47:21.752790 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260f0c35-383b-4548-8543-65354dda3ec0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:21.753074 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260f0c35-383b-4548-8543-65354dda3ec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:21.753351 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260f0c35-383b-4548-8543-65354dda3ec0 admin admin] PATH_INFO: `/v3/projects/96b405511a7a4849983a086f7bf243be/users/41d74daf53344ce2867e4018b32f8024/roles/3771341b3be641139e752aab690397a4` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:21.769099 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260f0c35-383b-4548-8543-65354dda3ec0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96b405511a7a4849983a086f7bf243be, user_id=41d74daf53344ce2867e4018b32f8024, role_id=3771341b3be641139e752aab690397a4)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:47:21.772639 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260f0c35-383b-4548-8543-65354dda3ec0 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:47:21.832495 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1212/2392] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:47:21 2022] PUT /identity/v3/projects/96b405511a7a4849983a086f7bf243be/users/41d74daf53344ce2867e4018b32f8024/roles/3771341b3be641139e752aab690397a4 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:47:21.843981 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df620a15-9ee8-4bb9-80dd-10dec6a77c6a None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:21.844279 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df620a15-9ee8-4bb9-80dd-10dec6a77c6a None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:21.844594 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df620a15-9ee8-4bb9-80dd-10dec6a77c6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:21.871067 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-df620a15-9ee8-4bb9-80dd-10dec6a77c6a None None] MFA Rules not processed for user `41d74daf53344ce2867e4018b32f8024`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:47:21.878230 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-df620a15-9ee8-4bb9-80dd-10dec6a77c6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:47:21.942921 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1181/2393] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:47:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:47:21.944827 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260f0c35-383b-4548-8543-65354dda3ec0 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:47:22.026103 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260f0c35-383b-4548-8543-65354dda3ec0 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:47:22.031064 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61322bb7-72c9-4716-974d-7f7fc10ca488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:47:22.031732 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61322bb7-72c9-4716-974d-7f7fc10ca488 admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:22.032019 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61322bb7-72c9-4716-974d-7f7fc10ca488 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:22.032302 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61322bb7-72c9-4716-974d-7f7fc10ca488 admin admin] PATH_INFO: `/v3/projects` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:22.033091 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61322bb7-72c9-4716-974d-7f7fc10ca488 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:47:22.035005 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61322bb7-72c9-4716-974d-7f7fc10ca488 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:47:22.159485 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1213/2394] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:47:21 2022] POST /identity/v3/projects => generated 385 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:47:22.163728 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df620a15-9ee8-4bb9-80dd-10dec6a77c6a None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:47:22.176514 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df620a15-9ee8-4bb9-80dd-10dec6a77c6a None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:47:22.239544 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fbe7a2-4038-4cd5-bdec-28fe36038aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:47:22.240214 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fbe7a2-4038-4cd5-bdec-28fe36038aeb admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:22.240530 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fbe7a2-4038-4cd5-bdec-28fe36038aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:22.240809 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fbe7a2-4038-4cd5-bdec-28fe36038aeb admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:22.241493 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fbe7a2-4038-4cd5-bdec-28fe36038aeb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:47:22.243382 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fbe7a2-4038-4cd5-bdec-28fe36038aeb admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:47:22.322184 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1182/2395] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:47:22 2022] POST /identity/v3/users => generated 299 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:47:22.326394 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61322bb7-72c9-4716-974d-7f7fc10ca488 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:47:22.340402 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61322bb7-72c9-4716-974d-7f7fc10ca488 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:47:22.347953 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8114270-8858-4b36-b124-8fad9dc3f54d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:47:22.348835 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8114270-8858-4b36-b124-8fad9dc3f54d admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:22.349125 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8114270-8858-4b36-b124-8fad9dc3f54d admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:22.349422 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8114270-8858-4b36-b124-8fad9dc3f54d admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:22.350155 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8114270-8858-4b36-b124-8fad9dc3f54d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:47:22.351769 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8114270-8858-4b36-b124-8fad9dc3f54d admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:47:22.363755 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1214/2396] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:47:22 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:47:22.367906 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fbe7a2-4038-4cd5-bdec-28fe36038aeb admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:47:22.378244 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fbe7a2-4038-4cd5-bdec-28fe36038aeb admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:47:22.382971 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a1a3a1-4dff-4177-bef3-d08173a41e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:47:22.383856 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a1a3a1-4dff-4177-bef3-d08173a41e65 admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:22.384146 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a1a3a1-4dff-4177-bef3-d08173a41e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:22.384481 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a1a3a1-4dff-4177-bef3-d08173a41e65 admin admin] PATH_INFO: `/v3/projects/14d9ddf8728d4b3e971e51f0b3575ea6/users/bfc8104a40744448a539c69975d148f7/roles/9baff2b18f2649f7b757ff2e1f63c853` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:22.400263 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a1a3a1-4dff-4177-bef3-d08173a41e65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14d9ddf8728d4b3e971e51f0b3575ea6, user_id=bfc8104a40744448a539c69975d148f7, role_id=9baff2b18f2649f7b757ff2e1f63c853)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:47:22.402955 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a1a3a1-4dff-4177-bef3-d08173a41e65 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:47:22.466232 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:47:22.472971 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1183/2397] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:47:22 2022] PUT /identity/v3/projects/14d9ddf8728d4b3e971e51f0b3575ea6/users/bfc8104a40744448a539c69975d148f7/roles/9baff2b18f2649f7b757ff2e1f63c853 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:47:22.477029 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8114270-8858-4b36-b124-8fad9dc3f54d admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:47:22.487245 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8114270-8858-4b36-b124-8fad9dc3f54d admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:47:22.545120 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d286a3ba-98fb-4777-aecf-29790e2b2905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:47:22.545802 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d286a3ba-98fb-4777-aecf-29790e2b2905 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:22.546118 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d286a3ba-98fb-4777-aecf-29790e2b2905 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:22.546403 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d286a3ba-98fb-4777-aecf-29790e2b2905 admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:22.547153 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d286a3ba-98fb-4777-aecf-29790e2b2905 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:47:22.548853 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d286a3ba-98fb-4777-aecf-29790e2b2905 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:47:22.558750 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1215/2398] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:47:22 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:47:22.563123 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a1a3a1-4dff-4177-bef3-d08173a41e65 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:47:22.576275 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a1a3a1-4dff-4177-bef3-d08173a41e65 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:47:22.583798 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17b0956-48dc-41fe-a212-55b3ad2df82b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:47:22.584987 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b0956-48dc-41fe-a212-55b3ad2df82b admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:22.585466 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b0956-48dc-41fe-a212-55b3ad2df82b admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:22.585744 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b0956-48dc-41fe-a212-55b3ad2df82b admin admin] PATH_INFO: `/v3/projects/14d9ddf8728d4b3e971e51f0b3575ea6/users/bfc8104a40744448a539c69975d148f7/roles/9baff2b18f2649f7b757ff2e1f63c853` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:22.587798 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17b0956-48dc-41fe-a212-55b3ad2df82b admin admin] RBAC: Authorizing `identity:create_grant(project_id=14d9ddf8728d4b3e971e51f0b3575ea6, user_id=bfc8104a40744448a539c69975d148f7, role_id=9baff2b18f2649f7b757ff2e1f63c853)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:47:22.590445 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17b0956-48dc-41fe-a212-55b3ad2df82b admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:47:22.603693 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1184/2399] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:47:22 2022] PUT /identity/v3/projects/14d9ddf8728d4b3e971e51f0b3575ea6/users/bfc8104a40744448a539c69975d148f7/roles/9baff2b18f2649f7b757ff2e1f63c853 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:47:22.613877 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2788e135-6f1d-4683-861e-ddcba62e07fc None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:22.614257 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2788e135-6f1d-4683-861e-ddcba62e07fc None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:22.614606 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2788e135-6f1d-4683-861e-ddcba62e07fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:22.648822 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-2788e135-6f1d-4683-861e-ddcba62e07fc None None] MFA Rules not processed for user `bfc8104a40744448a539c69975d148f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:47:22.650046 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-2788e135-6f1d-4683-861e-ddcba62e07fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:47:22.675141 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:47:22.720427 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1216/2400] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:47:22 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:47:22.732174 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17b0956-48dc-41fe-a212-55b3ad2df82b admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:47:22.733972 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-f17b0956-48dc-41fe-a212-55b3ad2df82b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:47:22.792753 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17b0956-48dc-41fe-a212-55b3ad2df82b admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:47:22.796869 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225f3752-3f0b-42d7-8714-15cae1bb966c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d81f597cb44c56a966537089bcce4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:47:22.797518 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225f3752-3f0b-42d7-8714-15cae1bb966c service cinder] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:22.797802 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225f3752-3f0b-42d7-8714-15cae1bb966c service cinder] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:22.798080 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225f3752-3f0b-42d7-8714-15cae1bb966c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:22.855389 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225f3752-3f0b-42d7-8714-15cae1bb966c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:47:22.856855 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225f3752-3f0b-42d7-8714-15cae1bb966c service cinder] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:47:22.866653 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1185/2401] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:47:22 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:47:30.207189 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2788e135-6f1d-4683-861e-ddcba62e07fc None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:47:30.208904 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-2788e135-6f1d-4683-861e-ddcba62e07fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:47:30.222184 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2788e135-6f1d-4683-861e-ddcba62e07fc None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:47:30.226675 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54b2750-b661-44bd-ba70-968da9501696 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d81f597cb44c56a966537089bcce4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:47:30.227412 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54b2750-b661-44bd-ba70-968da9501696 service cinder] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:30.227697 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54b2750-b661-44bd-ba70-968da9501696 service cinder] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:30.227975 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54b2750-b661-44bd-ba70-968da9501696 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:30.238913 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54b2750-b661-44bd-ba70-968da9501696 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:47:30.241432 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54b2750-b661-44bd-ba70-968da9501696 service cinder] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:47:30.251820 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1217/2402] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:47:30 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:47:32.861873 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=54,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:47:37.220022 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225f3752-3f0b-42d7-8714-15cae1bb966c service cinder] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:47:37.285718 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225f3752-3f0b-42d7-8714-15cae1bb966c service cinder] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:47:37.292191 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd59a5d6-2b03-43de-a91e-b343d4c6a516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:47:37.293095 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd59a5d6-2b03-43de-a91e-b343d4c6a516 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:37.293379 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd59a5d6-2b03-43de-a91e-b343d4c6a516 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:37.293654 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd59a5d6-2b03-43de-a91e-b343d4c6a516 admin admin] PATH_INFO: `/v3/users/41d74daf53344ce2867e4018b32f8024` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:37.294842 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd59a5d6-2b03-43de-a91e-b343d4c6a516 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41d74daf53344ce2867e4018b32f8024)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:47:37.296839 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd59a5d6-2b03-43de-a91e-b343d4c6a516 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:47:37.439619 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-fd59a5d6-2b03-43de-a91e-b343d4c6a516 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41d74daf53344ce2867e4018b32f8024'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:47:37.440283 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-fd59a5d6-2b03-43de-a91e-b343d4c6a516 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41d74daf53344ce2867e4018b32f8024'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:47:37.440923 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-fd59a5d6-2b03-43de-a91e-b343d4c6a516 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41d74daf53344ce2867e4018b32f8024'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:47:37.448260 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-fd59a5d6-2b03-43de-a91e-b343d4c6a516 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41d74daf53344ce2867e4018b32f8024'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:47:37.461649 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-fd59a5d6-2b03-43de-a91e-b343d4c6a516 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41d74daf53344ce2867e4018b32f8024'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:47:37.532695 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1186/2403] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:47:37 2022] DELETE /identity/v3/users/41d74daf53344ce2867e4018b32f8024 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:47:37.537001 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cf8a9b-9fb6-4bd2-946c-e08c4c71b0b7 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:37.538070 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cf8a9b-9fb6-4bd2-946c-e08c4c71b0b7 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:37.538070 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cf8a9b-9fb6-4bd2-946c-e08c4c71b0b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:37.557186 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-b9cf8a9b-9fb6-4bd2-946c-e08c4c71b0b7 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:47:37.558494 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-b9cf8a9b-9fb6-4bd2-946c-e08c4c71b0b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:47:37.624016 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1218/2404] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:47:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:47:37.634888 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd59a5d6-2b03-43de-a91e-b343d4c6a516 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:47:37.636725 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-fd59a5d6-2b03-43de-a91e-b343d4c6a516 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:47:37.709936 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd59a5d6-2b03-43de-a91e-b343d4c6a516 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:47:37.714518 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e625d9-f033-48ea-8921-8677cb44768d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:47:37.715339 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e625d9-f033-48ea-8921-8677cb44768d service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:37.715643 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e625d9-f033-48ea-8921-8677cb44768d service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:37.715992 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e625d9-f033-48ea-8921-8677cb44768d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:37.729296 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e625d9-f033-48ea-8921-8677cb44768d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:47:37.730768 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e625d9-f033-48ea-8921-8677cb44768d service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:47:37.744317 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1187/2405] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:47:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:47:38.159568 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cf8a9b-9fb6-4bd2-946c-e08c4c71b0b7 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:47:38.164134 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-b9cf8a9b-9fb6-4bd2-946c-e08c4c71b0b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:47:38.175004 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cf8a9b-9fb6-4bd2-946c-e08c4c71b0b7 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:47:38.180655 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b699c9-8909-4653-b212-219b7aa4c6ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:47:38.181782 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b699c9-8909-4653-b212-219b7aa4c6ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:38.182068 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b699c9-8909-4653-b212-219b7aa4c6ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:38.182349 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b699c9-8909-4653-b212-219b7aa4c6ee admin admin] PATH_INFO: `/v3/projects/96b405511a7a4849983a086f7bf243be` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:38.183912 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b699c9-8909-4653-b212-219b7aa4c6ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=96b405511a7a4849983a086f7bf243be)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:47:38.185839 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b699c9-8909-4653-b212-219b7aa4c6ee admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:47:38.294673 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-46b699c9-8909-4653-b212-219b7aa4c6ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96b405511a7a4849983a086f7bf243be', 'request_id': 'req-46b699c9-8909-4653-b212-219b7aa4c6ee'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:47:38.345557 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-46b699c9-8909-4653-b212-219b7aa4c6ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96b405511a7a4849983a086f7bf243be', 'request_id': 'req-46b699c9-8909-4653-b212-219b7aa4c6ee'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:47:38.351522 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-46b699c9-8909-4653-b212-219b7aa4c6ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96b405511a7a4849983a086f7bf243be', 'request_id': 'req-46b699c9-8909-4653-b212-219b7aa4c6ee'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:47:38.362767 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-46b699c9-8909-4653-b212-219b7aa4c6ee admin admin] The token cache is being invalidate because project 96b405511a7a4849983a086f7bf243be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:47:38.363057 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-46b699c9-8909-4653-b212-219b7aa4c6ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:47:38.363699 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-46b699c9-8909-4653-b212-219b7aa4c6ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:47:38.365287 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1219/2406] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:47:38 2022] DELETE /identity/v3/projects/96b405511a7a4849983a086f7bf243be => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:47:38.369386 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e625d9-f033-48ea-8921-8677cb44768d service neutron] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:47:38.379247 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e625d9-f033-48ea-8921-8677cb44768d service neutron] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:47:38.399559 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:47:38.437990 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cabcfd-9716-4241-8fd8-29b8cc309165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:47:38.438883 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cabcfd-9716-4241-8fd8-29b8cc309165 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:38.439178 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cabcfd-9716-4241-8fd8-29b8cc309165 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:38.439456 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cabcfd-9716-4241-8fd8-29b8cc309165 admin admin] PATH_INFO: `/v3/users/bfc8104a40744448a539c69975d148f7` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:38.440743 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cabcfd-9716-4241-8fd8-29b8cc309165 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfc8104a40744448a539c69975d148f7)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:47:38.442682 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cabcfd-9716-4241-8fd8-29b8cc309165 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:47:38.668336 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-86cabcfd-9716-4241-8fd8-29b8cc309165 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfc8104a40744448a539c69975d148f7'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:47:38.668968 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-86cabcfd-9716-4241-8fd8-29b8cc309165 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfc8104a40744448a539c69975d148f7'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:47:38.669604 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-86cabcfd-9716-4241-8fd8-29b8cc309165 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfc8104a40744448a539c69975d148f7'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:47:38.676792 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-86cabcfd-9716-4241-8fd8-29b8cc309165 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfc8104a40744448a539c69975d148f7'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:47:38.689682 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-86cabcfd-9716-4241-8fd8-29b8cc309165 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfc8104a40744448a539c69975d148f7'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:47:38.774552 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1188/2407] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:47:38 2022] DELETE /identity/v3/users/bfc8104a40744448a539c69975d148f7 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:47:39.158466 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b699c9-8909-4653-b212-219b7aa4c6ee admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:47:39.159656 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-46b699c9-8909-4653-b212-219b7aa4c6ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:47:39.173679 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b699c9-8909-4653-b212-219b7aa4c6ee admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:47:39.208256 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:47:39.246283 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643eb013-5625-4787-8b4d-513435597cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:47:39.247190 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643eb013-5625-4787-8b4d-513435597cd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:39.247469 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643eb013-5625-4787-8b4d-513435597cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:39.247749 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643eb013-5625-4787-8b4d-513435597cd8 admin admin] PATH_INFO: `/v3/projects/14d9ddf8728d4b3e971e51f0b3575ea6` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:39.248970 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643eb013-5625-4787-8b4d-513435597cd8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14d9ddf8728d4b3e971e51f0b3575ea6)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:47:39.250898 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643eb013-5625-4787-8b4d-513435597cd8 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:47:39.342291 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-643eb013-5625-4787-8b4d-513435597cd8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14d9ddf8728d4b3e971e51f0b3575ea6', 'request_id': 'req-643eb013-5625-4787-8b4d-513435597cd8'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:47:39.392359 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-643eb013-5625-4787-8b4d-513435597cd8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14d9ddf8728d4b3e971e51f0b3575ea6', 'request_id': 'req-643eb013-5625-4787-8b4d-513435597cd8'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:47:39.401684 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-643eb013-5625-4787-8b4d-513435597cd8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14d9ddf8728d4b3e971e51f0b3575ea6', 'request_id': 'req-643eb013-5625-4787-8b4d-513435597cd8'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:47:39.413751 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-643eb013-5625-4787-8b4d-513435597cd8 admin admin] The token cache is being invalidate because project 14d9ddf8728d4b3e971e51f0b3575ea6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:47:39.414040 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-643eb013-5625-4787-8b4d-513435597cd8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:47:39.414827 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-643eb013-5625-4787-8b4d-513435597cd8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:47:39.416268 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1220/2408] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:47:39 2022] DELETE /identity/v3/projects/14d9ddf8728d4b3e971e51f0b3575ea6 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:47:39.424067 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa62161-4570-4efa-9f56-b5a2fb0aae07 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:39.424067 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa62161-4570-4efa-9f56-b5a2fb0aae07 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:39.424067 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa62161-4570-4efa-9f56-b5a2fb0aae07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:39.440981 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-8fa62161-4570-4efa-9f56-b5a2fb0aae07 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:47:39.442154 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-8fa62161-4570-4efa-9f56-b5a2fb0aae07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:47:39.503603 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1189/2409] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:47:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:47:39.513026 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a9f171-7136-4cad-8ff5-ec57ad411606 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:39.513320 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a9f171-7136-4cad-8ff5-ec57ad411606 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:39.513607 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a9f171-7136-4cad-8ff5-ec57ad411606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:39.536324 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-d1a9f171-7136-4cad-8ff5-ec57ad411606 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:47:39.538174 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-d1a9f171-7136-4cad-8ff5-ec57ad411606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:47:39.546936 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1221/2410] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:47:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:47:39.551193 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa62161-4570-4efa-9f56-b5a2fb0aae07 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:47:39.561565 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa62161-4570-4efa-9f56-b5a2fb0aae07 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:47:39.567093 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9631a65d-e4e5-4ac7-ab50-cda4e60d4d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:47:39.567760 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9631a65d-e4e5-4ac7-ab50-cda4e60d4d84 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:39.568044 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9631a65d-e4e5-4ac7-ab50-cda4e60d4d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:39.568331 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9631a65d-e4e5-4ac7-ab50-cda4e60d4d84 admin admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:39.569097 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9631a65d-e4e5-4ac7-ab50-cda4e60d4d84 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:47:39.569410 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9631a65d-e4e5-4ac7-ab50-cda4e60d4d84 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:47:39.571037 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9631a65d-e4e5-4ac7-ab50-cda4e60d4d84 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:47:39.587491 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1190/2411] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jun 7 14:47:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:47:39.594325 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a9f171-7136-4cad-8ff5-ec57ad411606 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:47:39.605138 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a9f171-7136-4cad-8ff5-ec57ad411606 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:47:39.610070 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbb7809-a86b-4cb7-aed8-9dd4bd2b65ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:47:39.613575 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbb7809-a86b-4cb7-aed8-9dd4bd2b65ad admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:39.613575 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbb7809-a86b-4cb7-aed8-9dd4bd2b65ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:39.613575 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbb7809-a86b-4cb7-aed8-9dd4bd2b65ad admin admin] PATH_INFO: `/v3/projects` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:39.613575 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbb7809-a86b-4cb7-aed8-9dd4bd2b65ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:47:39.613971 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbb7809-a86b-4cb7-aed8-9dd4bd2b65ad admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:47:39.713800 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1222/2412] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:47:39 2022] POST /identity/v3/projects => generated 373 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:47:39.718007 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9631a65d-e4e5-4ac7-ab50-cda4e60d4d84 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:47:39.729296 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9631a65d-e4e5-4ac7-ab50-cda4e60d4d84 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:47:39.756539 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:47:39.786090 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54cec68-e4b3-4f5e-8eb9-4948a9781ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:47:39.786697 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54cec68-e4b3-4f5e-8eb9-4948a9781ad5 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:39.787058 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54cec68-e4b3-4f5e-8eb9-4948a9781ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:39.787847 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54cec68-e4b3-4f5e-8eb9-4948a9781ad5 admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:39.788591 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54cec68-e4b3-4f5e-8eb9-4948a9781ad5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:47:39.790661 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54cec68-e4b3-4f5e-8eb9-4948a9781ad5 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:47:39.912943 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1191/2413] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:47:39 2022] POST /identity/v3/users => generated 287 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:47:39.920047 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbb7809-a86b-4cb7-aed8-9dd4bd2b65ad admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:47:39.931943 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbb7809-a86b-4cb7-aed8-9dd4bd2b65ad admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:47:39.936655 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3577cf83-4388-4725-8654-16f05d84dbc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:47:39.946526 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3577cf83-4388-4725-8654-16f05d84dbc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:39.946526 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3577cf83-4388-4725-8654-16f05d84dbc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:39.946526 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3577cf83-4388-4725-8654-16f05d84dbc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:39.946526 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3577cf83-4388-4725-8654-16f05d84dbc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:47:39.946526 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3577cf83-4388-4725-8654-16f05d84dbc9 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:47:39.962534 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1223/2414] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:47:39 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:47:39.974104 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54cec68-e4b3-4f5e-8eb9-4948a9781ad5 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:47:39.977974 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54cec68-e4b3-4f5e-8eb9-4948a9781ad5 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:47:39.982813 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc3c673-547a-4c4e-a15d-255d6a4ae612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:47:39.983507 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc3c673-547a-4c4e-a15d-255d6a4ae612 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:39.983786 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc3c673-547a-4c4e-a15d-255d6a4ae612 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:39.984063 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc3c673-547a-4c4e-a15d-255d6a4ae612 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:39.984933 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc3c673-547a-4c4e-a15d-255d6a4ae612 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:47:39.986471 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc3c673-547a-4c4e-a15d-255d6a4ae612 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:47:39.998452 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1192/2415] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:47:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:47:40.002638 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3577cf83-4388-4725-8654-16f05d84dbc9 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:47:40.013301 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3577cf83-4388-4725-8654-16f05d84dbc9 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:47:40.018174 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a4e57e-0e6a-48ca-8f13-ec6150793a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:47:40.019074 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a4e57e-0e6a-48ca-8f13-ec6150793a00 admin admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:40.019358 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a4e57e-0e6a-48ca-8f13-ec6150793a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:40.019632 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a4e57e-0e6a-48ca-8f13-ec6150793a00 admin admin] PATH_INFO: `/v3/projects/6f578365f1be480c9339376560d824b3/users/da5f0c536abf4c3f94f4816566db469b/roles/3771341b3be641139e752aab690397a4` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:40.036865 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a4e57e-0e6a-48ca-8f13-ec6150793a00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f578365f1be480c9339376560d824b3, user_id=da5f0c536abf4c3f94f4816566db469b, role_id=3771341b3be641139e752aab690397a4)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:47:40.039956 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a4e57e-0e6a-48ca-8f13-ec6150793a00 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:47:40.106389 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1224/2416] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:47:40 2022] PUT /identity/v3/projects/6f578365f1be480c9339376560d824b3/users/da5f0c536abf4c3f94f4816566db469b/roles/3771341b3be641139e752aab690397a4 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:47:40.118662 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993766ad-41a2-4f33-8eb4-9718c46a6cb3 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:40.118966 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993766ad-41a2-4f33-8eb4-9718c46a6cb3 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:40.119230 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993766ad-41a2-4f33-8eb4-9718c46a6cb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:40.149464 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-993766ad-41a2-4f33-8eb4-9718c46a6cb3 None None] MFA Rules not processed for user `da5f0c536abf4c3f94f4816566db469b`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:47:40.150796 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-993766ad-41a2-4f33-8eb4-9718c46a6cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:47:40.228522 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1193/2417] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:47:40 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:47:40.239340 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a4e57e-0e6a-48ca-8f13-ec6150793a00 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:47:40.242128 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-49a4e57e-0e6a-48ca-8f13-ec6150793a00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:47:40.300005 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a4e57e-0e6a-48ca-8f13-ec6150793a00 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:47:40.304623 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a437e72-899d-48c1-bc11-4269be7af68f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d81f597cb44c56a966537089bcce4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:47:40.305281 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a437e72-899d-48c1-bc11-4269be7af68f service cinder] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:40.305563 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a437e72-899d-48c1-bc11-4269be7af68f service cinder] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:40.305841 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a437e72-899d-48c1-bc11-4269be7af68f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:40.314505 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a437e72-899d-48c1-bc11-4269be7af68f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:47:40.315964 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a437e72-899d-48c1-bc11-4269be7af68f service cinder] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:47:40.321150 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:47:40.326468 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1225/2418] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:47:40 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:47:45.378660 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993766ad-41a2-4f33-8eb4-9718c46a6cb3 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:47:45.442618 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993766ad-41a2-4f33-8eb4-9718c46a6cb3 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:47:45.447222 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8846ea9e-c4a7-4192-8321-77c6882288a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:47:45.448094 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8846ea9e-c4a7-4192-8321-77c6882288a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:45.448392 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8846ea9e-c4a7-4192-8321-77c6882288a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:45.448675 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8846ea9e-c4a7-4192-8321-77c6882288a6 admin admin] PATH_INFO: `/v3/users/da5f0c536abf4c3f94f4816566db469b` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:45.449883 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8846ea9e-c4a7-4192-8321-77c6882288a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da5f0c536abf4c3f94f4816566db469b)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:47:45.451791 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8846ea9e-c4a7-4192-8321-77c6882288a6 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:47:45.658254 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:47:45.666415 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-8846ea9e-c4a7-4192-8321-77c6882288a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da5f0c536abf4c3f94f4816566db469b'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:47:45.667498 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-8846ea9e-c4a7-4192-8321-77c6882288a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da5f0c536abf4c3f94f4816566db469b'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:47:45.668298 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-8846ea9e-c4a7-4192-8321-77c6882288a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da5f0c536abf4c3f94f4816566db469b'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:47:45.675755 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-8846ea9e-c4a7-4192-8321-77c6882288a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da5f0c536abf4c3f94f4816566db469b'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:47:45.688801 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-8846ea9e-c4a7-4192-8321-77c6882288a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da5f0c536abf4c3f94f4816566db469b'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:47:45.751237 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1194/2419] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:47:45 2022] DELETE /identity/v3/users/da5f0c536abf4c3f94f4816566db469b => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:47:45.756579 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b701c8d-4c41-4548-b824-e320c94142d3 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:45.757076 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b701c8d-4c41-4548-b824-e320c94142d3 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:45.757197 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b701c8d-4c41-4548-b824-e320c94142d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:45.775608 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-4b701c8d-4c41-4548-b824-e320c94142d3 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:47:45.776861 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-4b701c8d-4c41-4548-b824-e320c94142d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:47:45.836616 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1226/2420] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:47:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:47:45.846214 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8846ea9e-c4a7-4192-8321-77c6882288a6 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:47:45.847611 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-8846ea9e-c4a7-4192-8321-77c6882288a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:47:45.906295 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8846ea9e-c4a7-4192-8321-77c6882288a6 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:47:45.910095 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b9f280-01b8-4cc2-a614-0c16695d0958 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:47:45.910773 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b9f280-01b8-4cc2-a614-0c16695d0958 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:45.911052 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b9f280-01b8-4cc2-a614-0c16695d0958 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:45.911314 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b9f280-01b8-4cc2-a614-0c16695d0958 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:45.924394 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b9f280-01b8-4cc2-a614-0c16695d0958 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:47:45.925826 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b9f280-01b8-4cc2-a614-0c16695d0958 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:47:45.935347 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1195/2421] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:47:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:47:46.336852 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b701c8d-4c41-4548-b824-e320c94142d3 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:47:46.338460 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-4b701c8d-4c41-4548-b824-e320c94142d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:47:46.354410 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b701c8d-4c41-4548-b824-e320c94142d3 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:47:46.358947 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46372b85-a99e-4357-a9f5-512729f264e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:47:46.359841 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46372b85-a99e-4357-a9f5-512729f264e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:46.360123 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46372b85-a99e-4357-a9f5-512729f264e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:46.360445 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46372b85-a99e-4357-a9f5-512729f264e4 admin admin] PATH_INFO: `/v3/projects/6f578365f1be480c9339376560d824b3` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:46.361648 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46372b85-a99e-4357-a9f5-512729f264e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f578365f1be480c9339376560d824b3)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:47:46.363658 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46372b85-a99e-4357-a9f5-512729f264e4 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:47:46.479903 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-46372b85-a99e-4357-a9f5-512729f264e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f578365f1be480c9339376560d824b3', 'request_id': 'req-46372b85-a99e-4357-a9f5-512729f264e4'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:47:46.534214 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-46372b85-a99e-4357-a9f5-512729f264e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f578365f1be480c9339376560d824b3', 'request_id': 'req-46372b85-a99e-4357-a9f5-512729f264e4'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:47:46.541194 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-46372b85-a99e-4357-a9f5-512729f264e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f578365f1be480c9339376560d824b3', 'request_id': 'req-46372b85-a99e-4357-a9f5-512729f264e4'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:47:46.555566 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-46372b85-a99e-4357-a9f5-512729f264e4 admin admin] The token cache is being invalidate because project 6f578365f1be480c9339376560d824b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:47:46.555856 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-46372b85-a99e-4357-a9f5-512729f264e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:47:46.556559 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-46372b85-a99e-4357-a9f5-512729f264e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:47:46.558148 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1227/2422] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:47:46 2022] DELETE /identity/v3/projects/6f578365f1be480c9339376560d824b3 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:47:46.563233 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd1881f-037d-4f8e-9e08-8703dbf685c3 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:46.563863 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd1881f-037d-4f8e-9e08-8703dbf685c3 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:46.563863 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd1881f-037d-4f8e-9e08-8703dbf685c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:46.583578 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-ecd1881f-037d-4f8e-9e08-8703dbf685c3 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:47:46.584767 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-ecd1881f-037d-4f8e-9e08-8703dbf685c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:47:46.664832 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1196/2423] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:47:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:47:46.672894 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6addbe96-bd4a-4246-8579-e7d79c323d25 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:46.673299 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6addbe96-bd4a-4246-8579-e7d79c323d25 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:46.673509 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6addbe96-bd4a-4246-8579-e7d79c323d25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:46.696880 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-6addbe96-bd4a-4246-8579-e7d79c323d25 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:47:46.698456 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-6addbe96-bd4a-4246-8579-e7d79c323d25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:47:46.711100 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1228/2424] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:47:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:47:46.716666 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd1881f-037d-4f8e-9e08-8703dbf685c3 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:47:46.730457 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd1881f-037d-4f8e-9e08-8703dbf685c3 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:47:46.736153 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ee3400-993d-4057-b5b0-ae228134f2d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:47:46.743233 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ee3400-993d-4057-b5b0-ae228134f2d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:46.743233 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ee3400-993d-4057-b5b0-ae228134f2d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:46.743233 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ee3400-993d-4057-b5b0-ae228134f2d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:46.743233 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ee3400-993d-4057-b5b0-ae228134f2d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:47:46.743233 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ee3400-993d-4057-b5b0-ae228134f2d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:47:46.743233 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ee3400-993d-4057-b5b0-ae228134f2d2 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:47:46.753923 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1197/2425] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jun 7 14:47:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:47:46.759095 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6addbe96-bd4a-4246-8579-e7d79c323d25 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:47:46.768567 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6addbe96-bd4a-4246-8579-e7d79c323d25 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:47:46.773934 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61785d86-e478-4de9-89e6-8d1e8a5f7cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:47:46.774599 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61785d86-e478-4de9-89e6-8d1e8a5f7cdc admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:46.774882 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61785d86-e478-4de9-89e6-8d1e8a5f7cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:46.775158 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61785d86-e478-4de9-89e6-8d1e8a5f7cdc admin admin] PATH_INFO: `/v3/projects` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:46.775849 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61785d86-e478-4de9-89e6-8d1e8a5f7cdc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:47:46.777788 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61785d86-e478-4de9-89e6-8d1e8a5f7cdc admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:47:46.878549 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1229/2426] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:47:46 2022] POST /identity/v3/projects => generated 375 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:47:46.883471 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ee3400-993d-4057-b5b0-ae228134f2d2 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:47:46.900389 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ee3400-993d-4057-b5b0-ae228134f2d2 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:47:46.948838 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:47:46.965633 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14daa466-0d56-4ade-8196-e78c0dbdc620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:47:46.965633 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14daa466-0d56-4ade-8196-e78c0dbdc620 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:46.965633 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14daa466-0d56-4ade-8196-e78c0dbdc620 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:46.965633 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14daa466-0d56-4ade-8196-e78c0dbdc620 admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:46.966046 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14daa466-0d56-4ade-8196-e78c0dbdc620 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:47:46.966579 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14daa466-0d56-4ade-8196-e78c0dbdc620 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:47:47.077408 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1198/2427] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:47:46 2022] POST /identity/v3/users => generated 288 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:47:47.081759 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61785d86-e478-4de9-89e6-8d1e8a5f7cdc admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:47:47.093589 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61785d86-e478-4de9-89e6-8d1e8a5f7cdc admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:47:47.099754 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40b75ac-a0d1-44ac-b249-a37da389f029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:47:47.100940 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40b75ac-a0d1-44ac-b249-a37da389f029 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:47.101433 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40b75ac-a0d1-44ac-b249-a37da389f029 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:47.101916 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40b75ac-a0d1-44ac-b249-a37da389f029 admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:47.103298 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40b75ac-a0d1-44ac-b249-a37da389f029 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:47:47.106437 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40b75ac-a0d1-44ac-b249-a37da389f029 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:47:47.119781 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1230/2428] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:47:47 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:47:47.123903 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14daa466-0d56-4ade-8196-e78c0dbdc620 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:47:47.135156 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14daa466-0d56-4ade-8196-e78c0dbdc620 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:47:47.140087 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6ea924-0253-4565-acfc-b7a54683064a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:47:47.141299 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6ea924-0253-4565-acfc-b7a54683064a admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:47.141299 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6ea924-0253-4565-acfc-b7a54683064a admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:47.141480 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6ea924-0253-4565-acfc-b7a54683064a admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:47.142195 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6ea924-0253-4565-acfc-b7a54683064a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:47:47.143758 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6ea924-0253-4565-acfc-b7a54683064a admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:47:47.154241 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1199/2429] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:47:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:47:47.158690 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40b75ac-a0d1-44ac-b249-a37da389f029 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:47:47.170739 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40b75ac-a0d1-44ac-b249-a37da389f029 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:47:47.177225 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234b2066-0d68-4af3-991a-bcbc16c44bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:47:47.178978 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234b2066-0d68-4af3-991a-bcbc16c44bad admin admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:47.179268 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234b2066-0d68-4af3-991a-bcbc16c44bad admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:47.179549 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234b2066-0d68-4af3-991a-bcbc16c44bad admin admin] PATH_INFO: `/v3/projects/efd2a03022c141c9b2c200a353ae3843/users/884bca6091bd4eacab68279e52ab264a/roles/3771341b3be641139e752aab690397a4` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:47.195652 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234b2066-0d68-4af3-991a-bcbc16c44bad admin admin] RBAC: Authorizing `identity:create_grant(project_id=efd2a03022c141c9b2c200a353ae3843, user_id=884bca6091bd4eacab68279e52ab264a, role_id=3771341b3be641139e752aab690397a4)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:47:47.198372 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234b2066-0d68-4af3-991a-bcbc16c44bad admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:47:47.303732 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1231/2430] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:47:47 2022] PUT /identity/v3/projects/efd2a03022c141c9b2c200a353ae3843/users/884bca6091bd4eacab68279e52ab264a/roles/3771341b3be641139e752aab690397a4 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:47:47.311028 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91247cd6-475a-4b32-b80b-6534ffc52fc3 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:47.311318 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91247cd6-475a-4b32-b80b-6534ffc52fc3 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:47.311594 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91247cd6-475a-4b32-b80b-6534ffc52fc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:47.342564 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-91247cd6-475a-4b32-b80b-6534ffc52fc3 None None] MFA Rules not processed for user `884bca6091bd4eacab68279e52ab264a`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:47:47.343840 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-91247cd6-475a-4b32-b80b-6534ffc52fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:47:47.422596 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1200/2431] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:47:47 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:47:47.433700 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234b2066-0d68-4af3-991a-bcbc16c44bad admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:47:47.435487 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-234b2066-0d68-4af3-991a-bcbc16c44bad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:47:47.457894 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:47:47.493343 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234b2066-0d68-4af3-991a-bcbc16c44bad admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:47:47.497409 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d175af8c-eac8-4cd9-a5e0-7c9dcbd3a14b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d81f597cb44c56a966537089bcce4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:47:47.498187 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d175af8c-eac8-4cd9-a5e0-7c9dcbd3a14b service cinder] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:47.498481 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d175af8c-eac8-4cd9-a5e0-7c9dcbd3a14b service cinder] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:47.498762 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d175af8c-eac8-4cd9-a5e0-7c9dcbd3a14b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:47.508464 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d175af8c-eac8-4cd9-a5e0-7c9dcbd3a14b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:47:47.509969 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d175af8c-eac8-4cd9-a5e0-7c9dcbd3a14b service cinder] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:47:47.520422 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1232/2432] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:47:47 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:47:57.419338 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=45,keystone:INSERT=3 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:47:57.779017 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=17 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:47:58.864991 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91247cd6-475a-4b32-b80b-6534ffc52fc3 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:47:58.929308 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91247cd6-475a-4b32-b80b-6534ffc52fc3 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:47:58.934333 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6609edc6-9840-4e74-9c7f-70e7cbbfbb93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:47:58.935205 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6609edc6-9840-4e74-9c7f-70e7cbbfbb93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:58.935491 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6609edc6-9840-4e74-9c7f-70e7cbbfbb93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:58.935817 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6609edc6-9840-4e74-9c7f-70e7cbbfbb93 admin admin] PATH_INFO: `/v3/users/884bca6091bd4eacab68279e52ab264a` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:58.937140 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6609edc6-9840-4e74-9c7f-70e7cbbfbb93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=884bca6091bd4eacab68279e52ab264a)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:47:58.939168 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6609edc6-9840-4e74-9c7f-70e7cbbfbb93 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:47:59.095438 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-6609edc6-9840-4e74-9c7f-70e7cbbfbb93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '884bca6091bd4eacab68279e52ab264a'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:47:59.096106 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-6609edc6-9840-4e74-9c7f-70e7cbbfbb93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '884bca6091bd4eacab68279e52ab264a'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:47:59.096771 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-6609edc6-9840-4e74-9c7f-70e7cbbfbb93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '884bca6091bd4eacab68279e52ab264a'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:47:59.104431 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-6609edc6-9840-4e74-9c7f-70e7cbbfbb93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '884bca6091bd4eacab68279e52ab264a'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:47:59.117134 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-6609edc6-9840-4e74-9c7f-70e7cbbfbb93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '884bca6091bd4eacab68279e52ab264a'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:47:59.219881 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1201/2433] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:47:58 2022] DELETE /identity/v3/users/884bca6091bd4eacab68279e52ab264a => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:47:59.224558 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87715b2f-d1c4-4ede-86f2-fb0424ea4df8 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:59.231463 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87715b2f-d1c4-4ede-86f2-fb0424ea4df8 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:59.231463 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87715b2f-d1c4-4ede-86f2-fb0424ea4df8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:59.244663 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-87715b2f-d1c4-4ede-86f2-fb0424ea4df8 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:47:59.245700 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-87715b2f-d1c4-4ede-86f2-fb0424ea4df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:47:59.307179 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1233/2434] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:47:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:47:59.317030 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6609edc6-9840-4e74-9c7f-70e7cbbfbb93 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:47:59.318505 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-6609edc6-9840-4e74-9c7f-70e7cbbfbb93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:47:59.377351 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6609edc6-9840-4e74-9c7f-70e7cbbfbb93 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:47:59.381237 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fe5a76-997a-4925-b3de-bf5f108ea70d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:47:59.382003 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fe5a76-997a-4925-b3de-bf5f108ea70d service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:59.382361 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fe5a76-997a-4925-b3de-bf5f108ea70d service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:59.382712 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fe5a76-997a-4925-b3de-bf5f108ea70d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:59.393012 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fe5a76-997a-4925-b3de-bf5f108ea70d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:47:59.395751 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fe5a76-997a-4925-b3de-bf5f108ea70d service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:47:59.405089 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1202/2435] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:47:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:47:59.708000 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f732335e-a1f6-4000-9169-3a5d5db9b1c0 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:59.708966 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f732335e-a1f6-4000-9169-3a5d5db9b1c0 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:59.708966 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f732335e-a1f6-4000-9169-3a5d5db9b1c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:59.728153 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-f732335e-a1f6-4000-9169-3a5d5db9b1c0 None None] MFA Rules not processed for user `34eb9ec1f3ff44bbb03cf934cb2a04de`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:47:59.729397 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-f732335e-a1f6-4000-9169-3a5d5db9b1c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:47:59.857651 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fe5a76-997a-4925-b3de-bf5f108ea70d service neutron] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:47:59.859211 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-94fe5a76-997a-4925-b3de-bf5f108ea70d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:47:59.873665 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fe5a76-997a-4925-b3de-bf5f108ea70d service neutron] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:47:59.879201 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc2ce57-ac9c-4535-b36a-eba37b34b2a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:47:59.883829 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc2ce57-ac9c-4535-b36a-eba37b34b2a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:47:59.883829 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc2ce57-ac9c-4535-b36a-eba37b34b2a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:47:59.883829 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc2ce57-ac9c-4535-b36a-eba37b34b2a1 admin admin] PATH_INFO: `/v3/projects/efd2a03022c141c9b2c200a353ae3843` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:47:59.883829 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc2ce57-ac9c-4535-b36a-eba37b34b2a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=efd2a03022c141c9b2c200a353ae3843)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:47:59.884180 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc2ce57-ac9c-4535-b36a-eba37b34b2a1 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:47:59.904450 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:47:59.925539 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1234/2436] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:47:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:48:00.009542 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-8dc2ce57-ac9c-4535-b36a-eba37b34b2a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efd2a03022c141c9b2c200a353ae3843', 'request_id': 'req-8dc2ce57-ac9c-4535-b36a-eba37b34b2a1'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:48:00.093343 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-8dc2ce57-ac9c-4535-b36a-eba37b34b2a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efd2a03022c141c9b2c200a353ae3843', 'request_id': 'req-8dc2ce57-ac9c-4535-b36a-eba37b34b2a1'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:48:00.099800 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-8dc2ce57-ac9c-4535-b36a-eba37b34b2a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efd2a03022c141c9b2c200a353ae3843', 'request_id': 'req-8dc2ce57-ac9c-4535-b36a-eba37b34b2a1'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:48:00.111689 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-8dc2ce57-ac9c-4535-b36a-eba37b34b2a1 admin admin] The token cache is being invalidate because project efd2a03022c141c9b2c200a353ae3843 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:48:00.113964 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-8dc2ce57-ac9c-4535-b36a-eba37b34b2a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:48:00.113964 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-8dc2ce57-ac9c-4535-b36a-eba37b34b2a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:48:00.114238 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1203/2437] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:47:59 2022] DELETE /identity/v3/projects/efd2a03022c141c9b2c200a353ae3843 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:48:00.120404 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca4d37-8d65-46db-b4dc-26c87d0a332d None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:48:00.120701 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca4d37-8d65-46db-b4dc-26c87d0a332d None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:48:00.121034 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca4d37-8d65-46db-b4dc-26c87d0a332d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:48:00.139336 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-4fca4d37-8d65-46db-b4dc-26c87d0a332d None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:48:00.148457 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-4fca4d37-8d65-46db-b4dc-26c87d0a332d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:48:00.205141 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1235/2438] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:48:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:48:00.213208 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7426711b-3a8d-452b-8945-bb84e1d4b721 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:48:00.213508 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7426711b-3a8d-452b-8945-bb84e1d4b721 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:48:00.213790 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7426711b-3a8d-452b-8945-bb84e1d4b721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:48:00.233129 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-7426711b-3a8d-452b-8945-bb84e1d4b721 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:48:00.234345 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-7426711b-3a8d-452b-8945-bb84e1d4b721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:48:00.242513 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1204/2439] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:48:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:48:00.246944 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fca4d37-8d65-46db-b4dc-26c87d0a332d None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:48:00.256837 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fca4d37-8d65-46db-b4dc-26c87d0a332d None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:48:00.261838 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149b78d0-eb01-4d8c-9f44-a61f7ebb8ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:48:00.262503 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149b78d0-eb01-4d8c-9f44-a61f7ebb8ceb admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:48:00.262788 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149b78d0-eb01-4d8c-9f44-a61f7ebb8ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:48:00.263066 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149b78d0-eb01-4d8c-9f44-a61f7ebb8ceb admin admin] PATH_INFO: `/v3/domains` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:48:00.263829 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149b78d0-eb01-4d8c-9f44-a61f7ebb8ceb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:48:00.264143 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149b78d0-eb01-4d8c-9f44-a61f7ebb8ceb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:48:00.265779 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149b78d0-eb01-4d8c-9f44-a61f7ebb8ceb admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:48:00.278826 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1236/2440] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jun 7 14:48:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:48:00.282900 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7426711b-3a8d-452b-8945-bb84e1d4b721 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:48:00.292886 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7426711b-3a8d-452b-8945-bb84e1d4b721 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:48:00.297878 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bb62c8-b139-4c9d-ae0b-0f9e89b23b52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:48:00.299024 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bb62c8-b139-4c9d-ae0b-0f9e89b23b52 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:48:00.299024 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bb62c8-b139-4c9d-ae0b-0f9e89b23b52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:48:00.299279 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bb62c8-b139-4c9d-ae0b-0f9e89b23b52 admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:48:00.299931 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bb62c8-b139-4c9d-ae0b-0f9e89b23b52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:48:00.301876 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bb62c8-b139-4c9d-ae0b-0f9e89b23b52 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:48:00.438933 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1205/2441] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:48:00 2022] POST /identity/v3/projects => generated 375 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:48:00.443146 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149b78d0-eb01-4d8c-9f44-a61f7ebb8ceb admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:48:00.453177 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149b78d0-eb01-4d8c-9f44-a61f7ebb8ceb admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:48:00.484536 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:48:00.512620 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52474053-802f-4fc6-87f7-5d56ae0cab70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:48:00.513316 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52474053-802f-4fc6-87f7-5d56ae0cab70 admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:48:00.513595 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52474053-802f-4fc6-87f7-5d56ae0cab70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:48:00.513894 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52474053-802f-4fc6-87f7-5d56ae0cab70 admin admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:48:00.514755 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52474053-802f-4fc6-87f7-5d56ae0cab70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:48:00.516715 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52474053-802f-4fc6-87f7-5d56ae0cab70 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:48:00.666506 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1237/2442] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:48:00 2022] POST /identity/v3/users => generated 288 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:48:00.669539 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bb62c8-b139-4c9d-ae0b-0f9e89b23b52 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:48:00.682594 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bb62c8-b139-4c9d-ae0b-0f9e89b23b52 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:48:00.688644 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da722aa0-1879-4d8d-9efc-dd69badcae00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:48:00.689332 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da722aa0-1879-4d8d-9efc-dd69badcae00 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:48:00.689626 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da722aa0-1879-4d8d-9efc-dd69badcae00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:48:00.689911 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da722aa0-1879-4d8d-9efc-dd69badcae00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:48:00.690643 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da722aa0-1879-4d8d-9efc-dd69badcae00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:48:00.692265 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da722aa0-1879-4d8d-9efc-dd69badcae00 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:48:00.704239 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1206/2443] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:48:00 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:48:00.708185 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52474053-802f-4fc6-87f7-5d56ae0cab70 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:48:00.719380 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52474053-802f-4fc6-87f7-5d56ae0cab70 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:48:00.724623 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3956ff7-5170-472a-9b95-835e53c04731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:48:00.725297 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3956ff7-5170-472a-9b95-835e53c04731 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:48:00.725593 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3956ff7-5170-472a-9b95-835e53c04731 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:48:00.725987 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3956ff7-5170-472a-9b95-835e53c04731 admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:48:00.726600 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3956ff7-5170-472a-9b95-835e53c04731 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:48:00.728412 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3956ff7-5170-472a-9b95-835e53c04731 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:48:00.747246 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1238/2444] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:48:00 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 14:48:00.751481 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da722aa0-1879-4d8d-9efc-dd69badcae00 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:48:00.766421 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da722aa0-1879-4d8d-9efc-dd69badcae00 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:48:00.771029 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe682c6-5716-4a3c-9b83-3124b7482cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:48:00.771978 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe682c6-5716-4a3c-9b83-3124b7482cbe admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:48:00.772267 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe682c6-5716-4a3c-9b83-3124b7482cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:48:00.772654 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe682c6-5716-4a3c-9b83-3124b7482cbe admin admin] PATH_INFO: `/v3/projects/4828df5e016c42bfa253b68f732f99cb/users/2684a7ce412b4124af2155c516b69966/roles/3771341b3be641139e752aab690397a4` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:48:00.789141 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe682c6-5716-4a3c-9b83-3124b7482cbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=4828df5e016c42bfa253b68f732f99cb, user_id=2684a7ce412b4124af2155c516b69966, role_id=3771341b3be641139e752aab690397a4)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:48:00.791813 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe682c6-5716-4a3c-9b83-3124b7482cbe admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:48:00.918097 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1207/2445] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:48:00 2022] PUT /identity/v3/projects/4828df5e016c42bfa253b68f732f99cb/users/2684a7ce412b4124af2155c516b69966/roles/3771341b3be641139e752aab690397a4 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:48:00.929391 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c935505-ea5e-46a5-82bf-06428e935c1a None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:48:00.929709 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c935505-ea5e-46a5-82bf-06428e935c1a None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:48:00.929998 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c935505-ea5e-46a5-82bf-06428e935c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:48:00.957703 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-2c935505-ea5e-46a5-82bf-06428e935c1a None None] MFA Rules not processed for user `2684a7ce412b4124af2155c516b69966`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:48:00.958906 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-2c935505-ea5e-46a5-82bf-06428e935c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:48:01.021927 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1239/2446] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:48:00 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:48:01.034212 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe682c6-5716-4a3c-9b83-3124b7482cbe admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:48:01.035912 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-fbe682c6-5716-4a3c-9b83-3124b7482cbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:48:01.098548 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe682c6-5716-4a3c-9b83-3124b7482cbe admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:48:01.102208 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575cf777-78e7-4c52-8e7d-ca897987ac3a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d81f597cb44c56a966537089bcce4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:48:01.102856 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575cf777-78e7-4c52-8e7d-ca897987ac3a service cinder] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:48:01.103143 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575cf777-78e7-4c52-8e7d-ca897987ac3a service cinder] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:48:01.103422 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575cf777-78e7-4c52-8e7d-ca897987ac3a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:48:01.112999 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575cf777-78e7-4c52-8e7d-ca897987ac3a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:48:01.114488 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575cf777-78e7-4c52-8e7d-ca897987ac3a service cinder] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:48:01.123575 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1208/2447] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:48:01 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 14:48:03.987488 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287209b3-1aea-465a-8919-50a9f5ede634 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:48:03.988025 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287209b3-1aea-465a-8919-50a9f5ede634 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:48:03.988025 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287209b3-1aea-465a-8919-50a9f5ede634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:48:04.021769 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-287209b3-1aea-465a-8919-50a9f5ede634 None None] MFA Rules not processed for user `2684a7ce412b4124af2155c516b69966`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:48:04.023341 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-287209b3-1aea-465a-8919-50a9f5ede634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:48:04.031713 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1240/2448] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:48:03 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:48:11.120745 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=70,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:48:14.008890 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=54,keystone:INSERT=3 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:48:25.235616 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a41e73a-b267-42cd-a898-825af7a88229 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:48:25.236098 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a41e73a-b267-42cd-a898-825af7a88229 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:48:25.239866 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a41e73a-b267-42cd-a898-825af7a88229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:48:25.253993 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-7a41e73a-b267-42cd-a898-825af7a88229 None None] MFA Rules not processed for user `ad9d021b171244d796c320aa9f8450a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:48:25.255188 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-7a41e73a-b267-42cd-a898-825af7a88229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:48:25.310811 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1209/2449] 10.222.0.85 () {58 vars in 1009 bytes} [Tue Jun 7 14:48:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:48:25.458301 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287209b3-1aea-465a-8919-50a9f5ede634 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:48:25.459708 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-287209b3-1aea-465a-8919-50a9f5ede634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:48:25.527088 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287209b3-1aea-465a-8919-50a9f5ede634 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:48:25.532556 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ee9fee-f1c1-40b2-8c17-b70850020555 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35aa67025f66459da0c0fc3cbfd45430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:48:25.533211 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ee9fee-f1c1-40b2-8c17-b70850020555 service swift] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:48:25.533501 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ee9fee-f1c1-40b2-8c17-b70850020555 service swift] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:48:25.533782 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ee9fee-f1c1-40b2-8c17-b70850020555 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:48:25.542956 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ee9fee-f1c1-40b2-8c17-b70850020555 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:48:25.544563 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ee9fee-f1c1-40b2-8c17-b70850020555 service swift] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:48:25.552548 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1241/2450] 10.222.0.85 () {60 vars in 1384 bytes} [Tue Jun 7 14:48:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 14:48:26.342931 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64a2336-c63a-4664-926b-f2095593bb74 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:48:26.343401 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64a2336-c63a-4664-926b-f2095593bb74 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:48:26.343495 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64a2336-c63a-4664-926b-f2095593bb74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:48:26.361201 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-b64a2336-c63a-4664-926b-f2095593bb74 None None] MFA Rules not processed for user `ad9d021b171244d796c320aa9f8450a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:48:26.362393 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-b64a2336-c63a-4664-926b-f2095593bb74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:48:26.372096 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1210/2451] 10.222.0.85 () {58 vars in 967 bytes} [Tue Jun 7 14:48:26 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:48:26.382155 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ee9fee-f1c1-40b2-8c17-b70850020555 service swift] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:48:26.393014 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ee9fee-f1c1-40b2-8c17-b70850020555 service swift] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:48:26.397919 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812877c7-9ca3-4f03-bf0d-16bde423bf03 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35aa67025f66459da0c0fc3cbfd45430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:48:26.398801 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812877c7-9ca3-4f03-bf0d-16bde423bf03 service swift] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:48:26.399080 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812877c7-9ca3-4f03-bf0d-16bde423bf03 service swift] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:48:26.399355 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812877c7-9ca3-4f03-bf0d-16bde423bf03 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:48:26.407786 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812877c7-9ca3-4f03-bf0d-16bde423bf03 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:48:26.409376 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812877c7-9ca3-4f03-bf0d-16bde423bf03 service swift] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:48:26.417135 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1242/2452] 10.222.0.85 () {60 vars in 1384 bytes} [Tue Jun 7 14:48:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 14:48:27.111918 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f8267a-ad51-4020-9746-fb853ad022c3 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:48:27.116093 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f8267a-ad51-4020-9746-fb853ad022c3 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:48:27.116093 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f8267a-ad51-4020-9746-fb853ad022c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:48:27.132292 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-b1f8267a-ad51-4020-9746-fb853ad022c3 None None] MFA Rules not processed for user `ad9d021b171244d796c320aa9f8450a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:48:27.133539 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-b1f8267a-ad51-4020-9746-fb853ad022c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:48:27.141415 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1211/2453] 10.222.0.85 () {58 vars in 1009 bytes} [Tue Jun 7 14:48:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:48:27.154961 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812877c7-9ca3-4f03-bf0d-16bde423bf03 service swift] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:48:27.166703 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812877c7-9ca3-4f03-bf0d-16bde423bf03 service swift] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:48:27.171740 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde2d4f4-2e48-4bc9-9848-f39bbe78d9e7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35aa67025f66459da0c0fc3cbfd45430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:48:27.172393 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde2d4f4-2e48-4bc9-9848-f39bbe78d9e7 service swift] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:48:27.172682 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde2d4f4-2e48-4bc9-9848-f39bbe78d9e7 service swift] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:48:27.172959 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde2d4f4-2e48-4bc9-9848-f39bbe78d9e7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:48:27.181087 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde2d4f4-2e48-4bc9-9848-f39bbe78d9e7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:48:27.182636 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde2d4f4-2e48-4bc9-9848-f39bbe78d9e7 service swift] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:48:27.191947 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1243/2454] 10.222.0.85 () {60 vars in 1384 bytes} [Tue Jun 7 14:48:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 07 14:48:27.690750 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18c6a6b-59ad-40a6-9e8e-10f9f851a029 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:48:27.691238 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18c6a6b-59ad-40a6-9e8e-10f9f851a029 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:48:27.691345 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18c6a6b-59ad-40a6-9e8e-10f9f851a029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:48:27.709766 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-b18c6a6b-59ad-40a6-9e8e-10f9f851a029 None None] MFA Rules not processed for user `ad9d021b171244d796c320aa9f8450a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:48:27.710944 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-b18c6a6b-59ad-40a6-9e8e-10f9f851a029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:48:27.718805 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1212/2455] 10.222.0.85 () {58 vars in 967 bytes} [Tue Jun 7 14:48:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:48:27.730802 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde2d4f4-2e48-4bc9-9848-f39bbe78d9e7 service swift] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:48:27.741314 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde2d4f4-2e48-4bc9-9848-f39bbe78d9e7 service swift] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:48:27.746312 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468005f4-3ec3-4e37-bfc2-9df8b1b003b5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35aa67025f66459da0c0fc3cbfd45430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:48:27.746941 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468005f4-3ec3-4e37-bfc2-9df8b1b003b5 service swift] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:48:27.747227 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468005f4-3ec3-4e37-bfc2-9df8b1b003b5 service swift] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:48:27.747506 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468005f4-3ec3-4e37-bfc2-9df8b1b003b5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:48:27.755598 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468005f4-3ec3-4e37-bfc2-9df8b1b003b5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:48:27.757193 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468005f4-3ec3-4e37-bfc2-9df8b1b003b5 service swift] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:48:27.765657 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1244/2456] 10.222.0.85 () {60 vars in 1384 bytes} [Tue Jun 7 14:48:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 14:48:30.348389 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18c6a6b-59ad-40a6-9e8e-10f9f851a029 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:48:30.409888 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18c6a6b-59ad-40a6-9e8e-10f9f851a029 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:48:30.414357 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8471401f-186c-4caa-adbe-677b83591e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:48:30.415218 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8471401f-186c-4caa-adbe-677b83591e48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:48:30.415501 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8471401f-186c-4caa-adbe-677b83591e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:48:30.415794 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8471401f-186c-4caa-adbe-677b83591e48 admin admin] PATH_INFO: `/v3/users/2684a7ce412b4124af2155c516b69966` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:48:30.416968 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8471401f-186c-4caa-adbe-677b83591e48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2684a7ce412b4124af2155c516b69966)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:48:30.419471 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8471401f-186c-4caa-adbe-677b83591e48 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:48:30.650279 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-8471401f-186c-4caa-adbe-677b83591e48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2684a7ce412b4124af2155c516b69966'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:48:30.650890 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-8471401f-186c-4caa-adbe-677b83591e48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2684a7ce412b4124af2155c516b69966'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:48:30.651495 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-8471401f-186c-4caa-adbe-677b83591e48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2684a7ce412b4124af2155c516b69966'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:48:30.658908 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-8471401f-186c-4caa-adbe-677b83591e48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2684a7ce412b4124af2155c516b69966'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:48:30.670646 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-8471401f-186c-4caa-adbe-677b83591e48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2684a7ce412b4124af2155c516b69966'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:48:30.739343 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1213/2457] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:48:30 2022] DELETE /identity/v3/users/2684a7ce412b4124af2155c516b69966 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:48:30.743873 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ba307-53c4-4a38-bf9f-624884e9a6fc None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:48:30.744354 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ba307-53c4-4a38-bf9f-624884e9a6fc None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:48:30.744498 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ba307-53c4-4a38-bf9f-624884e9a6fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:48:30.762804 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-4f9ba307-53c4-4a38-bf9f-624884e9a6fc None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:48:30.763991 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-4f9ba307-53c4-4a38-bf9f-624884e9a6fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:48:30.820811 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1245/2458] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:48:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:48:30.830825 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8471401f-186c-4caa-adbe-677b83591e48 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:48:30.832243 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-8471401f-186c-4caa-adbe-677b83591e48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:48:30.838782 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:48:30.889735 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8471401f-186c-4caa-adbe-677b83591e48 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:48:30.893607 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18b59b2-f60d-438b-b33e-2f16be52bdf5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:48:30.894809 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18b59b2-f60d-438b-b33e-2f16be52bdf5 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:48:30.895158 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18b59b2-f60d-438b-b33e-2f16be52bdf5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:48:30.895441 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18b59b2-f60d-438b-b33e-2f16be52bdf5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:48:30.905889 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18b59b2-f60d-438b-b33e-2f16be52bdf5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:48:30.907240 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18b59b2-f60d-438b-b33e-2f16be52bdf5 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:48:30.916962 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1214/2459] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:48:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:48:31.232121 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9ba307-53c4-4a38-bf9f-624884e9a6fc None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:48:31.233596 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-4f9ba307-53c4-4a38-bf9f-624884e9a6fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:48:31.247570 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9ba307-53c4-4a38-bf9f-624884e9a6fc None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:48:31.252791 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cdf2ac-ab33-43ad-b528-640574aeff76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:48:31.253661 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cdf2ac-ab33-43ad-b528-640574aeff76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:48:31.253947 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cdf2ac-ab33-43ad-b528-640574aeff76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:48:31.254224 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cdf2ac-ab33-43ad-b528-640574aeff76 admin admin] PATH_INFO: `/v3/projects/4828df5e016c42bfa253b68f732f99cb` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:48:31.255479 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cdf2ac-ab33-43ad-b528-640574aeff76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4828df5e016c42bfa253b68f732f99cb)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:48:31.257362 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cdf2ac-ab33-43ad-b528-640574aeff76 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:48:31.366186 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:48:31.369091 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-32cdf2ac-ab33-43ad-b528-640574aeff76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4828df5e016c42bfa253b68f732f99cb', 'request_id': 'req-32cdf2ac-ab33-43ad-b528-640574aeff76'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:48:31.464902 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-32cdf2ac-ab33-43ad-b528-640574aeff76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4828df5e016c42bfa253b68f732f99cb', 'request_id': 'req-32cdf2ac-ab33-43ad-b528-640574aeff76'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:48:31.471223 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-32cdf2ac-ab33-43ad-b528-640574aeff76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4828df5e016c42bfa253b68f732f99cb', 'request_id': 'req-32cdf2ac-ab33-43ad-b528-640574aeff76'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:48:31.483568 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-32cdf2ac-ab33-43ad-b528-640574aeff76 admin admin] The token cache is being invalidate because project 4828df5e016c42bfa253b68f732f99cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:48:31.483805 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-32cdf2ac-ab33-43ad-b528-640574aeff76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:48:31.484437 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-32cdf2ac-ab33-43ad-b528-640574aeff76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:48:31.485972 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1246/2460] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:48:31 2022] DELETE /identity/v3/projects/4828df5e016c42bfa253b68f732f99cb => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:48:31.491333 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e294b51-5e02-442e-a1bc-b617b72b8808 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:48:31.491815 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e294b51-5e02-442e-a1bc-b617b72b8808 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:48:31.491938 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e294b51-5e02-442e-a1bc-b617b72b8808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:48:31.510165 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-8e294b51-5e02-442e-a1bc-b617b72b8808 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:48:31.511397 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-8e294b51-5e02-442e-a1bc-b617b72b8808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:48:31.578908 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1215/2461] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:48:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:48:31.595817 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b22d8a5-19ad-4f10-9b54-aa3aa732e231 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:48:31.596095 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b22d8a5-19ad-4f10-9b54-aa3aa732e231 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:48:31.596416 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b22d8a5-19ad-4f10-9b54-aa3aa732e231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:48:31.615956 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-5b22d8a5-19ad-4f10-9b54-aa3aa732e231 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:48:31.617247 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-5b22d8a5-19ad-4f10-9b54-aa3aa732e231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:48:31.628266 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1247/2462] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:48:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:48:31.632959 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e294b51-5e02-442e-a1bc-b617b72b8808 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:48:31.644316 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e294b51-5e02-442e-a1bc-b617b72b8808 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:48:31.649369 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cfbbaf-02e5-46dd-935a-e1602feb6c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:48:31.650031 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cfbbaf-02e5-46dd-935a-e1602feb6c2e admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:48:31.650317 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cfbbaf-02e5-46dd-935a-e1602feb6c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:48:31.650599 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cfbbaf-02e5-46dd-935a-e1602feb6c2e admin admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:48:31.651327 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cfbbaf-02e5-46dd-935a-e1602feb6c2e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:48:31.651636 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cfbbaf-02e5-46dd-935a-e1602feb6c2e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:48:31.653914 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cfbbaf-02e5-46dd-935a-e1602feb6c2e admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:48:31.667901 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1216/2463] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jun 7 14:48:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:48:31.672964 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b22d8a5-19ad-4f10-9b54-aa3aa732e231 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:48:31.682217 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b22d8a5-19ad-4f10-9b54-aa3aa732e231 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:48:31.688609 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826dc065-fe14-432a-b7a6-f53bbc1ad938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:48:31.689256 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826dc065-fe14-432a-b7a6-f53bbc1ad938 admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:48:31.689540 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826dc065-fe14-432a-b7a6-f53bbc1ad938 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:48:31.689824 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826dc065-fe14-432a-b7a6-f53bbc1ad938 admin admin] PATH_INFO: `/v3/projects` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:48:31.690519 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826dc065-fe14-432a-b7a6-f53bbc1ad938 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:48:31.692455 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826dc065-fe14-432a-b7a6-f53bbc1ad938 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:48:31.758083 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1248/2464] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:48:31 2022] POST /identity/v3/projects => generated 383 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:48:31.764282 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cfbbaf-02e5-46dd-935a-e1602feb6c2e admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:48:31.776278 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cfbbaf-02e5-46dd-935a-e1602feb6c2e admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:48:31.850058 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb57aa5-278b-4f0e-970e-c7caa468902b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:48:31.850923 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb57aa5-278b-4f0e-970e-c7caa468902b admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:48:31.851245 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb57aa5-278b-4f0e-970e-c7caa468902b admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:48:31.851611 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb57aa5-278b-4f0e-970e-c7caa468902b admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:48:31.852579 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb57aa5-278b-4f0e-970e-c7caa468902b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:48:31.855154 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb57aa5-278b-4f0e-970e-c7caa468902b admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:48:31.974033 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1217/2465] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:48:31 2022] POST /identity/v3/users => generated 292 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:48:31.979684 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826dc065-fe14-432a-b7a6-f53bbc1ad938 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:48:31.991173 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826dc065-fe14-432a-b7a6-f53bbc1ad938 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:48:31.996659 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d9bfec-a523-4864-9c7f-5600b19c77db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:48:31.998149 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d9bfec-a523-4864-9c7f-5600b19c77db admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:48:31.998440 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d9bfec-a523-4864-9c7f-5600b19c77db admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:48:31.998770 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d9bfec-a523-4864-9c7f-5600b19c77db admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:48:31.999795 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d9bfec-a523-4864-9c7f-5600b19c77db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:48:32.001458 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d9bfec-a523-4864-9c7f-5600b19c77db admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:48:32.014858 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1249/2466] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:48:31 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:48:32.019129 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb57aa5-278b-4f0e-970e-c7caa468902b admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:48:32.031134 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb57aa5-278b-4f0e-970e-c7caa468902b admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:48:32.035973 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71c64d5-1da8-45ce-a309-0e3738b432bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:48:32.036735 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71c64d5-1da8-45ce-a309-0e3738b432bf admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:48:32.037269 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71c64d5-1da8-45ce-a309-0e3738b432bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:48:32.037546 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71c64d5-1da8-45ce-a309-0e3738b432bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:48:32.038279 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71c64d5-1da8-45ce-a309-0e3738b432bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:48:32.039855 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71c64d5-1da8-45ce-a309-0e3738b432bf admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:48:32.050396 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1218/2467] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:48:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:48:32.054917 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d9bfec-a523-4864-9c7f-5600b19c77db admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:48:32.065042 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d9bfec-a523-4864-9c7f-5600b19c77db admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:48:32.069549 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7aa155-cc69-4749-8210-ddf6d41b020c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:48:32.070664 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7aa155-cc69-4749-8210-ddf6d41b020c admin admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:48:32.070949 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7aa155-cc69-4749-8210-ddf6d41b020c admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:48:32.071225 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7aa155-cc69-4749-8210-ddf6d41b020c admin admin] PATH_INFO: `/v3/projects/70b63567b3e54cf1a4348fb65b65a040/users/f2b3f5c080034b658cbfb311c0c77b26/roles/3771341b3be641139e752aab690397a4` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:48:32.089005 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7aa155-cc69-4749-8210-ddf6d41b020c admin admin] RBAC: Authorizing `identity:create_grant(project_id=70b63567b3e54cf1a4348fb65b65a040, user_id=f2b3f5c080034b658cbfb311c0c77b26, role_id=3771341b3be641139e752aab690397a4)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:48:32.092319 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7aa155-cc69-4749-8210-ddf6d41b020c admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:48:32.175090 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1250/2468] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:48:32 2022] PUT /identity/v3/projects/70b63567b3e54cf1a4348fb65b65a040/users/f2b3f5c080034b658cbfb311c0c77b26/roles/3771341b3be641139e752aab690397a4 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:48:32.180208 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9403c5bf-fcfa-42c8-a64a-3e2038552207 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:48:32.181853 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9403c5bf-fcfa-42c8-a64a-3e2038552207 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:48:32.182141 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9403c5bf-fcfa-42c8-a64a-3e2038552207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:48:32.202912 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-9403c5bf-fcfa-42c8-a64a-3e2038552207 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:48:32.204591 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-9403c5bf-fcfa-42c8-a64a-3e2038552207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:48:32.217077 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:48:32.280123 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1219/2469] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:48:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:48:32.289947 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7aa155-cc69-4749-8210-ddf6d41b020c admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:48:32.291658 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-fb7aa155-cc69-4749-8210-ddf6d41b020c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:48:32.373196 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7aa155-cc69-4749-8210-ddf6d41b020c admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:48:32.379002 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15830c4-9408-43ae-9cd8-d451bf307c7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:48:32.379682 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15830c4-9408-43ae-9cd8-d451bf307c7b service neutron] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:48:32.380041 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15830c4-9408-43ae-9cd8-d451bf307c7b service neutron] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:48:32.380313 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15830c4-9408-43ae-9cd8-d451bf307c7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:48:32.390828 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15830c4-9408-43ae-9cd8-d451bf307c7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:48:32.392273 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15830c4-9408-43ae-9cd8-d451bf307c7b service neutron] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:48:32.409071 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1251/2470] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:48:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:48:37.946897 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e00ee78-5177-4a96-9ddc-43243a295698 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:48:37.951912 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e00ee78-5177-4a96-9ddc-43243a295698 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:48:37.952300 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e00ee78-5177-4a96-9ddc-43243a295698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:48:37.995829 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-4e00ee78-5177-4a96-9ddc-43243a295698 None None] MFA Rules not processed for user `f2b3f5c080034b658cbfb311c0c77b26`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:48:37.997702 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-4e00ee78-5177-4a96-9ddc-43243a295698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:48:38.098521 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1220/2471] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:48:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:48:38.102929 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15830c4-9408-43ae-9cd8-d451bf307c7b service neutron] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:48:38.113992 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15830c4-9408-43ae-9cd8-d451bf307c7b service neutron] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:48:38.119010 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438293b4-2f7f-4f9d-a012-dc426591ab0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:48:38.119679 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438293b4-2f7f-4f9d-a012-dc426591ab0e admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:48:38.119965 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438293b4-2f7f-4f9d-a012-dc426591ab0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:48:38.120244 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438293b4-2f7f-4f9d-a012-dc426591ab0e admin admin] PATH_INFO: `/v3/projects` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:48:38.121004 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438293b4-2f7f-4f9d-a012-dc426591ab0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:48:38.123525 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438293b4-2f7f-4f9d-a012-dc426591ab0e admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:48:38.210284 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1252/2472] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:48:38 2022] POST /identity/v3/projects => generated 383 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:48:38.214346 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e00ee78-5177-4a96-9ddc-43243a295698 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:48:38.229258 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e00ee78-5177-4a96-9ddc-43243a295698 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:48:38.319138 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0174c4-4105-4420-b2c2-006e5d34f031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:48:38.320547 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0174c4-4105-4420-b2c2-006e5d34f031 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:48:38.321048 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0174c4-4105-4420-b2c2-006e5d34f031 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:48:38.327716 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0174c4-4105-4420-b2c2-006e5d34f031 admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:48:38.327716 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0174c4-4105-4420-b2c2-006e5d34f031 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:48:38.327716 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0174c4-4105-4420-b2c2-006e5d34f031 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:48:38.443454 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1221/2473] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:48:38 2022] POST /identity/v3/users => generated 298 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:48:38.447361 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438293b4-2f7f-4f9d-a012-dc426591ab0e admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:48:38.461132 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438293b4-2f7f-4f9d-a012-dc426591ab0e admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:48:38.466392 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b700284d-31e4-4c52-bca2-e8422cc9ef6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:48:38.467084 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b700284d-31e4-4c52-bca2-e8422cc9ef6b admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:48:38.467365 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b700284d-31e4-4c52-bca2-e8422cc9ef6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:48:38.467639 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b700284d-31e4-4c52-bca2-e8422cc9ef6b admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:48:38.468364 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b700284d-31e4-4c52-bca2-e8422cc9ef6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:48:38.470051 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b700284d-31e4-4c52-bca2-e8422cc9ef6b admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:48:38.481304 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1253/2474] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:48:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:48:38.485479 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0174c4-4105-4420-b2c2-006e5d34f031 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:48:38.495574 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0174c4-4105-4420-b2c2-006e5d34f031 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:48:38.501098 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b343b8ac-d22a-4605-a252-93772bfdf8f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:48:38.502277 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b343b8ac-d22a-4605-a252-93772bfdf8f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:48:38.502565 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b343b8ac-d22a-4605-a252-93772bfdf8f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:48:38.502852 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b343b8ac-d22a-4605-a252-93772bfdf8f7 admin admin] PATH_INFO: `/v3/projects/4a2f8209eba443bb94046ac2df18413d/users/9a3a6b2727574aa690f87819ffa52c89/roles/9baff2b18f2649f7b757ff2e1f63c853` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:48:38.518723 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b343b8ac-d22a-4605-a252-93772bfdf8f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a2f8209eba443bb94046ac2df18413d, user_id=9a3a6b2727574aa690f87819ffa52c89, role_id=9baff2b18f2649f7b757ff2e1f63c853)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:48:38.521433 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b343b8ac-d22a-4605-a252-93772bfdf8f7 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:48:38.602553 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1222/2475] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:48:38 2022] PUT /identity/v3/projects/4a2f8209eba443bb94046ac2df18413d/users/9a3a6b2727574aa690f87819ffa52c89/roles/9baff2b18f2649f7b757ff2e1f63c853 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:48:38.607111 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b700284d-31e4-4c52-bca2-e8422cc9ef6b admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:48:38.617405 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b700284d-31e4-4c52-bca2-e8422cc9ef6b admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:48:38.657638 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:48:38.676722 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c586c37-d282-4ed5-a096-7dceb83262e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:48:38.677402 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c586c37-d282-4ed5-a096-7dceb83262e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:48:38.677691 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c586c37-d282-4ed5-a096-7dceb83262e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:48:38.677977 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c586c37-d282-4ed5-a096-7dceb83262e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:48:38.678743 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c586c37-d282-4ed5-a096-7dceb83262e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:48:38.680349 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c586c37-d282-4ed5-a096-7dceb83262e7 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:48:38.690711 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1254/2476] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:48:38 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:48:38.695062 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b343b8ac-d22a-4605-a252-93772bfdf8f7 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:48:38.705354 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b343b8ac-d22a-4605-a252-93772bfdf8f7 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:48:38.710137 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c470f84-7674-434e-bf4c-94c37a253492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:48:38.711035 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c470f84-7674-434e-bf4c-94c37a253492 admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:48:38.711323 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c470f84-7674-434e-bf4c-94c37a253492 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:48:38.711600 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c470f84-7674-434e-bf4c-94c37a253492 admin admin] PATH_INFO: `/v3/projects/4a2f8209eba443bb94046ac2df18413d/users/9a3a6b2727574aa690f87819ffa52c89/roles/9baff2b18f2649f7b757ff2e1f63c853` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:48:38.713599 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c470f84-7674-434e-bf4c-94c37a253492 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a2f8209eba443bb94046ac2df18413d, user_id=9a3a6b2727574aa690f87819ffa52c89, role_id=9baff2b18f2649f7b757ff2e1f63c853)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:48:38.716780 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c470f84-7674-434e-bf4c-94c37a253492 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:48:38.720011 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:48:38.728238 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1223/2477] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:48:38 2022] PUT /identity/v3/projects/4a2f8209eba443bb94046ac2df18413d/users/9a3a6b2727574aa690f87819ffa52c89/roles/9baff2b18f2649f7b757ff2e1f63c853 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:48:44.995541 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07686f67-ca9e-4c5f-8f77-4151a4fd069c None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:48:44.996023 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07686f67-ca9e-4c5f-8f77-4151a4fd069c None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:48:44.996166 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07686f67-ca9e-4c5f-8f77-4151a4fd069c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:48:45.047231 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-07686f67-ca9e-4c5f-8f77-4151a4fd069c None None] MFA Rules not processed for user `9a3a6b2727574aa690f87819ffa52c89`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:48:45.048617 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-07686f67-ca9e-4c5f-8f77-4151a4fd069c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:48:45.122062 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1255/2478] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:48:44 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:48:45.135279 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c470f84-7674-434e-bf4c-94c37a253492 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:48:45.137124 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-8c470f84-7674-434e-bf4c-94c37a253492 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:48:45.195643 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c470f84-7674-434e-bf4c-94c37a253492 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:48:45.199375 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48644707-1b88-4e02-9659-0f8696a5fe58 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51da9867d9147f198b8077e953c4f9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:48:45.200103 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48644707-1b88-4e02-9659-0f8696a5fe58 service glance] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:48:45.200414 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48644707-1b88-4e02-9659-0f8696a5fe58 service glance] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:48:45.200696 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48644707-1b88-4e02-9659-0f8696a5fe58 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:48:45.256342 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48644707-1b88-4e02-9659-0f8696a5fe58 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:48:45.257793 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48644707-1b88-4e02-9659-0f8696a5fe58 service glance] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:48:45.266632 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1224/2479] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:48:45 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:48:45.465899 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8edbe3-40b8-4850-ae52-0f2f3fdd3299 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:48:45.466377 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8edbe3-40b8-4850-ae52-0f2f3fdd3299 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:48:45.466779 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8edbe3-40b8-4850-ae52-0f2f3fdd3299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:48:45.485265 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-fe8edbe3-40b8-4850-ae52-0f2f3fdd3299 None None] MFA Rules not processed for user `ad9d021b171244d796c320aa9f8450a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:48:45.486473 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-fe8edbe3-40b8-4850-ae52-0f2f3fdd3299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:48:45.544594 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1256/2480] 10.222.0.85 () {58 vars in 1009 bytes} [Tue Jun 7 14:48:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:48:45.556767 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48644707-1b88-4e02-9659-0f8696a5fe58 service glance] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:48:45.558234 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-48644707-1b88-4e02-9659-0f8696a5fe58 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:48:45.630951 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48644707-1b88-4e02-9659-0f8696a5fe58 service glance] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:48:45.636526 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdb4e64-2f88-4a99-8ace-f1510f988654 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35aa67025f66459da0c0fc3cbfd45430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:48:45.637736 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdb4e64-2f88-4a99-8ace-f1510f988654 service swift] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:48:45.638041 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdb4e64-2f88-4a99-8ace-f1510f988654 service swift] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:48:45.638322 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdb4e64-2f88-4a99-8ace-f1510f988654 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:48:45.646961 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdb4e64-2f88-4a99-8ace-f1510f988654 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:48:45.648568 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdb4e64-2f88-4a99-8ace-f1510f988654 service swift] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:48:45.657710 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1225/2481] 10.222.0.85 () {60 vars in 1384 bytes} [Tue Jun 7 14:48:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 07 14:48:47.354364 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8145b4dc-9c54-496b-9d13-0290c33b0766 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:48:47.356088 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8145b4dc-9c54-496b-9d13-0290c33b0766 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:48:47.356659 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8145b4dc-9c54-496b-9d13-0290c33b0766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:48:47.438833 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-8145b4dc-9c54-496b-9d13-0290c33b0766 None None] MFA Rules not processed for user `ad9d021b171244d796c320aa9f8450a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:48:47.440093 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-8145b4dc-9c54-496b-9d13-0290c33b0766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:48:47.462133 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1257/2482] 10.222.0.85 () {58 vars in 1009 bytes} [Tue Jun 7 14:48:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:48:47.478599 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdb4e64-2f88-4a99-8ace-f1510f988654 service swift] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:48:47.506979 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdb4e64-2f88-4a99-8ace-f1510f988654 service swift] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:48:47.536728 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc16754-e873-4a45-a7ac-dbbe474cf841 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35aa67025f66459da0c0fc3cbfd45430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:48:47.537433 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc16754-e873-4a45-a7ac-dbbe474cf841 service swift] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:48:47.537726 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc16754-e873-4a45-a7ac-dbbe474cf841 service swift] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:48:47.538012 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc16754-e873-4a45-a7ac-dbbe474cf841 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:48:47.560522 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc16754-e873-4a45-a7ac-dbbe474cf841 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:48:47.562154 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc16754-e873-4a45-a7ac-dbbe474cf841 service swift] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:48:47.593576 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1226/2483] 10.222.0.85 () {60 vars in 1384 bytes} [Tue Jun 7 14:48:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 14:48:57.423282 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=73 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:48:57.597407 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=88 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:48:57.928435 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d34bb4-6111-42ee-92a5-5eac35d68756 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:48:57.928734 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d34bb4-6111-42ee-92a5-5eac35d68756 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:48:57.929013 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d34bb4-6111-42ee-92a5-5eac35d68756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:48:57.961366 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-59d34bb4-6111-42ee-92a5-5eac35d68756 None None] MFA Rules not processed for user `f2b3f5c080034b658cbfb311c0c77b26`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:48:57.962598 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-59d34bb4-6111-42ee-92a5-5eac35d68756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:48:57.971058 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1258/2484] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:48:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:49:05.232491 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c0f2e-5150-45cb-b1f4-657b24f04ea7 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:49:05.233019 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c0f2e-5150-45cb-b1f4-657b24f04ea7 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:49:05.233516 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c0f2e-5150-45cb-b1f4-657b24f04ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:49:05.251899 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-481c0f2e-5150-45cb-b1f4-657b24f04ea7 None None] MFA Rules not processed for user `ad9d021b171244d796c320aa9f8450a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:49:05.253147 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-481c0f2e-5150-45cb-b1f4-657b24f04ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:49:05.267844 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1227/2485] 10.222.0.85 () {58 vars in 1009 bytes} [Tue Jun 7 14:49:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:49:05.281715 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d34bb4-6111-42ee-92a5-5eac35d68756 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:49:05.295330 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d34bb4-6111-42ee-92a5-5eac35d68756 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:49:05.300663 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca140ff-cbc6-432c-89a8-e9137bf992b8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35aa67025f66459da0c0fc3cbfd45430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:49:05.301524 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca140ff-cbc6-432c-89a8-e9137bf992b8 service swift] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:49:05.301814 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca140ff-cbc6-432c-89a8-e9137bf992b8 service swift] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:49:05.302089 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca140ff-cbc6-432c-89a8-e9137bf992b8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:49:05.314634 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca140ff-cbc6-432c-89a8-e9137bf992b8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:49:05.316364 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca140ff-cbc6-432c-89a8-e9137bf992b8 service swift] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:49:05.337727 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1259/2486] 10.222.0.85 () {60 vars in 1384 bytes} [Tue Jun 7 14:49:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 14:49:15.246164 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=3 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:49:15.321751 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=9 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:49:15.775982 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481c0f2e-5150-45cb-b1f4-657b24f04ea7 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:49:15.841125 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481c0f2e-5150-45cb-b1f4-657b24f04ea7 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:49:15.844806 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db9a799-bc79-40c1-ad06-85748e620fe4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:49:15.845459 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db9a799-bc79-40c1-ad06-85748e620fe4 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:49:15.845747 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db9a799-bc79-40c1-ad06-85748e620fe4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:49:15.846025 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db9a799-bc79-40c1-ad06-85748e620fe4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:49:15.847850 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-6db9a799-bc79-40c1-ad06-85748e620fe4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:49:15.917700 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db9a799-bc79-40c1-ad06-85748e620fe4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:49:15.919714 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db9a799-bc79-40c1-ad06-85748e620fe4 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:49:15.930669 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1228/2487] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:49:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:49:17.492503 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca140ff-cbc6-432c-89a8-e9137bf992b8 service swift] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:49:17.493987 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-bca140ff-cbc6-432c-89a8-e9137bf992b8 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:49:17.506821 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca140ff-cbc6-432c-89a8-e9137bf992b8 service swift] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:49:17.510713 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a840499-603c-496c-bdae-d118d41144c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:49:17.511597 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a840499-603c-496c-bdae-d118d41144c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:49:17.511704 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a840499-603c-496c-bdae-d118d41144c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:49:17.511955 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a840499-603c-496c-bdae-d118d41144c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:49:17.517711 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-6a840499-603c-496c-bdae-d118d41144c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:49:17.533107 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a840499-603c-496c-bdae-d118d41144c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:49:17.534652 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a840499-603c-496c-bdae-d118d41144c2 service neutron] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:49:17.549813 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1260/2488] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:49:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:49:20.821530 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc1e4ca-d4b4-47c8-a71e-395010cb28bc None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:49:20.822058 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc1e4ca-d4b4-47c8-a71e-395010cb28bc None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:49:20.822173 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc1e4ca-d4b4-47c8-a71e-395010cb28bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:49:20.842375 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-1bc1e4ca-d4b4-47c8-a71e-395010cb28bc None None] MFA Rules not processed for user `ad9d021b171244d796c320aa9f8450a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:49:20.846226 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-1bc1e4ca-d4b4-47c8-a71e-395010cb28bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:49:20.856730 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1229/2489] 10.222.0.85 () {58 vars in 1009 bytes} [Tue Jun 7 14:49:20 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:49:20.869502 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a840499-603c-496c-bdae-d118d41144c2 service neutron] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:49:20.884554 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a840499-603c-496c-bdae-d118d41144c2 service neutron] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:49:20.891091 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728a8186-df54-4d9c-b211-615bf24a8aa3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35aa67025f66459da0c0fc3cbfd45430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:49:20.892111 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a8186-df54-4d9c-b211-615bf24a8aa3 service swift] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:49:20.892678 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a8186-df54-4d9c-b211-615bf24a8aa3 service swift] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:49:20.893220 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a8186-df54-4d9c-b211-615bf24a8aa3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:49:20.905838 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728a8186-df54-4d9c-b211-615bf24a8aa3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:49:20.907575 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728a8186-df54-4d9c-b211-615bf24a8aa3 service swift] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:49:20.924935 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1261/2490] 10.222.0.85 () {60 vars in 1384 bytes} [Tue Jun 7 14:49:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 14:49:30.217102 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc1e4ca-d4b4-47c8-a71e-395010cb28bc None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:49:30.218500 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-1bc1e4ca-d4b4-47c8-a71e-395010cb28bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:49:30.276171 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc1e4ca-d4b4-47c8-a71e-395010cb28bc None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:49:30.279813 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0695ebe-8cdf-4be4-bbf2-5a17cde35f8f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d81f597cb44c56a966537089bcce4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:49:30.280490 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0695ebe-8cdf-4be4-bbf2-5a17cde35f8f service cinder] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:49:30.280778 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0695ebe-8cdf-4be4-bbf2-5a17cde35f8f service cinder] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:49:30.281062 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0695ebe-8cdf-4be4-bbf2-5a17cde35f8f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:49:30.282846 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-a0695ebe-8cdf-4be4-bbf2-5a17cde35f8f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:49:30.332595 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:49:30.344894 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0695ebe-8cdf-4be4-bbf2-5a17cde35f8f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:49:30.346347 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0695ebe-8cdf-4be4-bbf2-5a17cde35f8f service cinder] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:49:30.355998 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1230/2491] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:49:30 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:49:30.919463 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=12 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:49:35.866149 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728a8186-df54-4d9c-b211-615bf24a8aa3 service swift] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:49:35.875731 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728a8186-df54-4d9c-b211-615bf24a8aa3 service swift] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:49:35.879470 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61730846-63fa-44bb-a7e2-eba8b92f6e5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:49:35.880128 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61730846-63fa-44bb-a7e2-eba8b92f6e5e service neutron] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:49:35.880450 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61730846-63fa-44bb-a7e2-eba8b92f6e5e service neutron] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:49:35.880731 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61730846-63fa-44bb-a7e2-eba8b92f6e5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:49:35.882550 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-61730846-63fa-44bb-a7e2-eba8b92f6e5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:49:35.893153 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61730846-63fa-44bb-a7e2-eba8b92f6e5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:49:35.894706 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61730846-63fa-44bb-a7e2-eba8b92f6e5e service neutron] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:49:35.906355 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1262/2492] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:49:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:49:40.360807 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=5 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:49:44.129442 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe101802-0d90-4873-aa28-19778dc7268a None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:49:44.130164 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe101802-0d90-4873-aa28-19778dc7268a None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:49:44.130164 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe101802-0d90-4873-aa28-19778dc7268a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:49:44.148149 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-fe101802-0d90-4873-aa28-19778dc7268a None None] MFA Rules not processed for user `0a15f5fa82414bdf8ca3f6723c0a8d2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:49:44.149391 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-fe101802-0d90-4873-aa28-19778dc7268a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:49:44.157314 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1231/2493] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:49:44 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:49:44.485099 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61730846-63fa-44bb-a7e2-eba8b92f6e5e service neutron] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:49:44.493701 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61730846-63fa-44bb-a7e2-eba8b92f6e5e service neutron] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:49:44.497286 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b171c006-2623-42f7-8d81-a6351d142fc3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:49:44.497927 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b171c006-2623-42f7-8d81-a6351d142fc3 service neutron] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:49:44.498215 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b171c006-2623-42f7-8d81-a6351d142fc3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:49:44.498495 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b171c006-2623-42f7-8d81-a6351d142fc3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:49:44.500261 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-b171c006-2623-42f7-8d81-a6351d142fc3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:49:44.509642 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b171c006-2623-42f7-8d81-a6351d142fc3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:49:44.511050 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b171c006-2623-42f7-8d81-a6351d142fc3 service neutron] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:49:44.519269 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1263/2494] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:49:44 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:49:45.327762 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe101802-0d90-4873-aa28-19778dc7268a None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:49:45.329315 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-fe101802-0d90-4873-aa28-19778dc7268a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:49:45.408145 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74928cc-9ec2-469e-88b6-f77939cb1c6e None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:49:45.408607 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74928cc-9ec2-469e-88b6-f77939cb1c6e None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:49:45.408987 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74928cc-9ec2-469e-88b6-f77939cb1c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:49:45.419247 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe101802-0d90-4873-aa28-19778dc7268a None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:49:45.425006 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7eba8c-80a9-40f6-9ca6-08179104cd5d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6330294b1c8e4dcd85aad189f007b0aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:49:45.425665 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7eba8c-80a9-40f6-9ca6-08179104cd5d service placement] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:49:45.425946 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7eba8c-80a9-40f6-9ca6-08179104cd5d service placement] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:49:45.426226 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7eba8c-80a9-40f6-9ca6-08179104cd5d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:49:45.428644 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-9c7eba8c-80a9-40f6-9ca6-08179104cd5d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:49:45.431094 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-b74928cc-9ec2-469e-88b6-f77939cb1c6e None None] MFA Rules not processed for user `ad9d021b171244d796c320aa9f8450a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:49:45.432514 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-b74928cc-9ec2-469e-88b6-f77939cb1c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:49:45.442184 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1264/2495] 10.222.0.85 () {58 vars in 967 bytes} [Tue Jun 7 14:49:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:49:45.446728 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7eba8c-80a9-40f6-9ca6-08179104cd5d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:49:45.448429 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7eba8c-80a9-40f6-9ca6-08179104cd5d service placement] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:49:45.460413 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1232/2496] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:49:45 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:49:45.465546 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74928cc-9ec2-469e-88b6-f77939cb1c6e None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:49:45.479679 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74928cc-9ec2-469e-88b6-f77939cb1c6e None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:49:45.485675 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27222720-b50a-488e-bd2e-06c2226c26e9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35aa67025f66459da0c0fc3cbfd45430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:49:45.486456 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27222720-b50a-488e-bd2e-06c2226c26e9 service swift] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:49:45.486743 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27222720-b50a-488e-bd2e-06c2226c26e9 service swift] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:49:45.487022 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27222720-b50a-488e-bd2e-06c2226c26e9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:49:45.495800 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27222720-b50a-488e-bd2e-06c2226c26e9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:49:45.500832 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27222720-b50a-488e-bd2e-06c2226c26e9 service swift] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:49:45.507360 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1265/2497] 10.222.0.85 () {60 vars in 1384 bytes} [Tue Jun 7 14:49:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 14:49:55.460700 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=35 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:49:55.503585 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=21 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:49:56.459549 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7eba8c-80a9-40f6-9ca6-08179104cd5d service placement] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:49:56.465193 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7eba8c-80a9-40f6-9ca6-08179104cd5d service placement] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:49:56.469162 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabb54a9-b576-481d-8be6-84ec2a5de118 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:49:56.469809 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabb54a9-b576-481d-8be6-84ec2a5de118 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:49:56.470383 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabb54a9-b576-481d-8be6-84ec2a5de118 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:49:56.470383 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabb54a9-b576-481d-8be6-84ec2a5de118 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:49:56.479677 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabb54a9-b576-481d-8be6-84ec2a5de118 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:49:56.481093 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabb54a9-b576-481d-8be6-84ec2a5de118 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:49:56.491177 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1233/2498] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:49:56 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:49:57.069403 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27222720-b50a-488e-bd2e-06c2226c26e9 service swift] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:49:57.083560 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27222720-b50a-488e-bd2e-06c2226c26e9 service swift] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:49:57.088764 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d2ec61-983d-4c34-874d-c0baf8fae5ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:49:57.100541 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d2ec61-983d-4c34-874d-c0baf8fae5ae service neutron] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:49:57.100541 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d2ec61-983d-4c34-874d-c0baf8fae5ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:49:57.100541 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d2ec61-983d-4c34-874d-c0baf8fae5ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:49:57.100541 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-13d2ec61-983d-4c34-874d-c0baf8fae5ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:49:57.174722 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d2ec61-983d-4c34-874d-c0baf8fae5ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:49:57.176766 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d2ec61-983d-4c34-874d-c0baf8fae5ae service neutron] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:49:57.189166 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1266/2499] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:49:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:50:01.659044 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabb54a9-b576-481d-8be6-84ec2a5de118 service neutron] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:50:01.660556 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-aabb54a9-b576-481d-8be6-84ec2a5de118 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:50:01.681232 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabb54a9-b576-481d-8be6-84ec2a5de118 service neutron] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:50:01.690140 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbf8f23-dd04-4ff1-b78f-9a9ac421b041 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6330294b1c8e4dcd85aad189f007b0aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:50:01.691316 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbf8f23-dd04-4ff1-b78f-9a9ac421b041 service placement] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:01.691316 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbf8f23-dd04-4ff1-b78f-9a9ac421b041 service placement] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:01.691530 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbf8f23-dd04-4ff1-b78f-9a9ac421b041 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:01.694184 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-2fbf8f23-dd04-4ff1-b78f-9a9ac421b041 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:50:01.711254 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbf8f23-dd04-4ff1-b78f-9a9ac421b041 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:50:01.713255 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbf8f23-dd04-4ff1-b78f-9a9ac421b041 service placement] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:50:01.724416 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1234/2500] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:50:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:50:01.786436 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d2ec61-983d-4c34-874d-c0baf8fae5ae service neutron] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:50:01.788802 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-13d2ec61-983d-4c34-874d-c0baf8fae5ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:50:01.808139 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d2ec61-983d-4c34-874d-c0baf8fae5ae service neutron] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:50:01.813543 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5f85ec-441d-413e-9f43-029320ef539d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:50:01.814413 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5f85ec-441d-413e-9f43-029320ef539d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:01.814881 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5f85ec-441d-413e-9f43-029320ef539d admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:01.815153 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5f85ec-441d-413e-9f43-029320ef539d admin admin] PATH_INFO: `/v3/users/4fb4b0cf5f8b4274b787f327793964f3` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:01.830917 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5f85ec-441d-413e-9f43-029320ef539d admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fb4b0cf5f8b4274b787f327793964f3)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:50:01.832948 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5f85ec-441d-413e-9f43-029320ef539d admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:50:02.002418 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-8d5f85ec-441d-413e-9f43-029320ef539d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fb4b0cf5f8b4274b787f327793964f3'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:50:02.002418 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-8d5f85ec-441d-413e-9f43-029320ef539d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fb4b0cf5f8b4274b787f327793964f3'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:50:02.002866 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-8d5f85ec-441d-413e-9f43-029320ef539d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fb4b0cf5f8b4274b787f327793964f3'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:50:02.011145 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-8d5f85ec-441d-413e-9f43-029320ef539d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fb4b0cf5f8b4274b787f327793964f3'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:50:02.036632 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-8d5f85ec-441d-413e-9f43-029320ef539d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fb4b0cf5f8b4274b787f327793964f3'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:50:02.114956 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1267/2501] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:50:01 2022] DELETE /identity/v3/users/4fb4b0cf5f8b4274b787f327793964f3 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:50:02.473289 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbf8f23-dd04-4ff1-b78f-9a9ac421b041 service placement] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:50:02.491560 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbf8f23-dd04-4ff1-b78f-9a9ac421b041 service placement] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:50:02.615024 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143a7d74-34e4-4620-ab62-c0a62ec789d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:50:02.616000 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143a7d74-34e4-4620-ab62-c0a62ec789d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:02.616361 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143a7d74-34e4-4620-ab62-c0a62ec789d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:02.616753 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143a7d74-34e4-4620-ab62-c0a62ec789d3 admin admin] PATH_INFO: `/v3/projects/97ba26b86c7243dab8e6c6b84ed674e8` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:02.632533 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143a7d74-34e4-4620-ab62-c0a62ec789d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97ba26b86c7243dab8e6c6b84ed674e8)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:50:02.636891 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143a7d74-34e4-4620-ab62-c0a62ec789d3 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:50:02.785368 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-143a7d74-34e4-4620-ab62-c0a62ec789d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97ba26b86c7243dab8e6c6b84ed674e8', 'request_id': 'req-143a7d74-34e4-4620-ab62-c0a62ec789d3'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:50:02.870006 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-143a7d74-34e4-4620-ab62-c0a62ec789d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97ba26b86c7243dab8e6c6b84ed674e8', 'request_id': 'req-143a7d74-34e4-4620-ab62-c0a62ec789d3'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:50:02.879696 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-143a7d74-34e4-4620-ab62-c0a62ec789d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97ba26b86c7243dab8e6c6b84ed674e8', 'request_id': 'req-143a7d74-34e4-4620-ab62-c0a62ec789d3'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:50:02.894068 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-143a7d74-34e4-4620-ab62-c0a62ec789d3 admin admin] The token cache is being invalidate because project 97ba26b86c7243dab8e6c6b84ed674e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:50:02.894403 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-143a7d74-34e4-4620-ab62-c0a62ec789d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:50:02.895066 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-143a7d74-34e4-4620-ab62-c0a62ec789d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:50:02.896741 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1235/2502] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:50:02 2022] DELETE /identity/v3/projects/97ba26b86c7243dab8e6c6b84ed674e8 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:50:02.902885 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abee183-cbe7-4c34-b262-1e6650961eca None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:02.903413 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abee183-cbe7-4c34-b262-1e6650961eca None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:02.903847 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abee183-cbe7-4c34-b262-1e6650961eca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:02.931098 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-1abee183-cbe7-4c34-b262-1e6650961eca None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:50:02.933735 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-1abee183-cbe7-4c34-b262-1e6650961eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:50:02.993544 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:50:03.019075 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1268/2503] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:50:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:50:03.036571 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0922bba-dc70-4d71-858b-c503e34fc43d None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:03.036961 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0922bba-dc70-4d71-858b-c503e34fc43d None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:03.037250 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0922bba-dc70-4d71-858b-c503e34fc43d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:03.057555 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-c0922bba-dc70-4d71-858b-c503e34fc43d None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:50:03.058823 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-c0922bba-dc70-4d71-858b-c503e34fc43d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:50:03.067137 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1236/2504] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:50:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:50:03.071741 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abee183-cbe7-4c34-b262-1e6650961eca None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:50:03.084162 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abee183-cbe7-4c34-b262-1e6650961eca None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:50:03.089468 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380d95ce-936e-4dd9-b41d-7ba0647c27f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:50:03.091152 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380d95ce-936e-4dd9-b41d-7ba0647c27f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:03.091152 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380d95ce-936e-4dd9-b41d-7ba0647c27f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:03.091671 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380d95ce-936e-4dd9-b41d-7ba0647c27f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:03.091920 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380d95ce-936e-4dd9-b41d-7ba0647c27f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:50:03.092230 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380d95ce-936e-4dd9-b41d-7ba0647c27f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:50:03.093889 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380d95ce-936e-4dd9-b41d-7ba0647c27f3 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:50:03.106118 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1269/2505] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jun 7 14:50:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:50:03.110596 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0922bba-dc70-4d71-858b-c503e34fc43d None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:50:03.120323 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0922bba-dc70-4d71-858b-c503e34fc43d None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:50:03.125750 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1a1cb5-84b1-463b-9c09-3bbeea3670f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:50:03.126432 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1a1cb5-84b1-463b-9c09-3bbeea3670f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:03.126721 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1a1cb5-84b1-463b-9c09-3bbeea3670f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:03.127001 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1a1cb5-84b1-463b-9c09-3bbeea3670f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:03.127698 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1a1cb5-84b1-463b-9c09-3bbeea3670f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:50:03.129644 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1a1cb5-84b1-463b-9c09-3bbeea3670f9 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:50:03.238708 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1237/2506] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:50:03 2022] POST /identity/v3/projects => generated 375 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:50:03.242985 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380d95ce-936e-4dd9-b41d-7ba0647c27f3 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:50:03.254569 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380d95ce-936e-4dd9-b41d-7ba0647c27f3 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:50:03.320635 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d26409-c2c9-4b63-80db-3cde29d9f6fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:50:03.321693 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d26409-c2c9-4b63-80db-3cde29d9f6fa admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:03.322139 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d26409-c2c9-4b63-80db-3cde29d9f6fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:03.322585 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d26409-c2c9-4b63-80db-3cde29d9f6fa admin admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:03.323452 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d26409-c2c9-4b63-80db-3cde29d9f6fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:50:03.326528 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d26409-c2c9-4b63-80db-3cde29d9f6fa admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:50:03.395344 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1270/2507] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:50:03 2022] POST /identity/v3/users => generated 288 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:50:03.399670 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1a1cb5-84b1-463b-9c09-3bbeea3670f9 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:50:03.410323 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1a1cb5-84b1-463b-9c09-3bbeea3670f9 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:50:03.415205 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7045e614-e712-4afc-a489-8436bba21fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:50:03.415872 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7045e614-e712-4afc-a489-8436bba21fb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:03.416187 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7045e614-e712-4afc-a489-8436bba21fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:03.416475 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7045e614-e712-4afc-a489-8436bba21fb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:03.417210 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7045e614-e712-4afc-a489-8436bba21fb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:50:03.418835 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7045e614-e712-4afc-a489-8436bba21fb5 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:50:03.432453 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1238/2508] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:50:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:50:03.437320 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d26409-c2c9-4b63-80db-3cde29d9f6fa admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:50:03.450648 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d26409-c2c9-4b63-80db-3cde29d9f6fa admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:50:03.456794 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315a0a73-079c-413c-b2f6-cd1f608eda68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:50:03.457893 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a0a73-079c-413c-b2f6-cd1f608eda68 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:03.458386 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a0a73-079c-413c-b2f6-cd1f608eda68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:03.459955 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a0a73-079c-413c-b2f6-cd1f608eda68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:03.459955 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315a0a73-079c-413c-b2f6-cd1f608eda68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:50:03.461408 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315a0a73-079c-413c-b2f6-cd1f608eda68 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:50:03.472007 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1271/2509] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:50:03 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:50:03.476574 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7045e614-e712-4afc-a489-8436bba21fb5 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:50:03.487769 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7045e614-e712-4afc-a489-8436bba21fb5 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:50:03.492857 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0d3a1f-eb04-4bd7-9537-4718b181f9d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:50:03.493745 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0d3a1f-eb04-4bd7-9537-4718b181f9d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:03.494327 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0d3a1f-eb04-4bd7-9537-4718b181f9d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:03.494327 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0d3a1f-eb04-4bd7-9537-4718b181f9d2 admin admin] PATH_INFO: `/v3/projects/5134adfcb5844a02a6d9d46ba2113147/users/d0d18d877bf84ba3a32ca0ce13f89816/roles/3771341b3be641139e752aab690397a4` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:03.509351 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:50:03.512530 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0d3a1f-eb04-4bd7-9537-4718b181f9d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5134adfcb5844a02a6d9d46ba2113147, user_id=d0d18d877bf84ba3a32ca0ce13f89816, role_id=3771341b3be641139e752aab690397a4)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:50:03.517372 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0d3a1f-eb04-4bd7-9537-4718b181f9d2 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:50:03.595198 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1239/2510] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:50:03 2022] PUT /identity/v3/projects/5134adfcb5844a02a6d9d46ba2113147/users/d0d18d877bf84ba3a32ca0ce13f89816/roles/3771341b3be641139e752aab690397a4 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:50:03.605814 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14462f88-de0d-4530-a8b3-66d9eadb103d None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:03.606131 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14462f88-de0d-4530-a8b3-66d9eadb103d None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:03.606436 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14462f88-de0d-4530-a8b3-66d9eadb103d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:03.632895 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-14462f88-de0d-4530-a8b3-66d9eadb103d None None] MFA Rules not processed for user `d0d18d877bf84ba3a32ca0ce13f89816`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:50:03.637428 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-14462f88-de0d-4530-a8b3-66d9eadb103d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:50:03.701417 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1272/2511] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:50:03 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:50:03.705914 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0d3a1f-eb04-4bd7-9537-4718b181f9d2 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:50:03.768981 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0d3a1f-eb04-4bd7-9537-4718b181f9d2 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:50:03.773836 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdbcfa7-d175-41f4-9cde-6e78f54da2bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:50:03.774494 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdbcfa7-d175-41f4-9cde-6e78f54da2bd admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:03.774779 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdbcfa7-d175-41f4-9cde-6e78f54da2bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:03.775325 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdbcfa7-d175-41f4-9cde-6e78f54da2bd admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:03.776016 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdbcfa7-d175-41f4-9cde-6e78f54da2bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:50:03.777986 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdbcfa7-d175-41f4-9cde-6e78f54da2bd admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:50:03.884410 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1240/2512] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:50:03 2022] POST /identity/v3/projects => generated 377 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:50:03.890489 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14462f88-de0d-4530-a8b3-66d9eadb103d None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:50:03.901809 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14462f88-de0d-4530-a8b3-66d9eadb103d None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:50:03.953270 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:50:03.966879 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4e5cc5-f2b1-4081-9c21-1834c64426d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:50:03.967559 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4e5cc5-f2b1-4081-9c21-1834c64426d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:03.967841 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4e5cc5-f2b1-4081-9c21-1834c64426d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:03.968112 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4e5cc5-f2b1-4081-9c21-1834c64426d9 admin admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:03.968868 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4e5cc5-f2b1-4081-9c21-1834c64426d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:50:03.970900 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4e5cc5-f2b1-4081-9c21-1834c64426d9 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:50:04.063880 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1273/2513] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:50:03 2022] POST /identity/v3/users => generated 295 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:50:04.067906 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdbcfa7-d175-41f4-9cde-6e78f54da2bd admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:50:04.079231 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdbcfa7-d175-41f4-9cde-6e78f54da2bd admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:50:04.084451 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2130ad00-5373-4e28-a441-dedf237af2b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:50:04.085131 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2130ad00-5373-4e28-a441-dedf237af2b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:04.085418 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2130ad00-5373-4e28-a441-dedf237af2b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:04.085699 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2130ad00-5373-4e28-a441-dedf237af2b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:04.086462 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2130ad00-5373-4e28-a441-dedf237af2b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:50:04.088083 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2130ad00-5373-4e28-a441-dedf237af2b8 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:50:04.099404 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1241/2514] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:50:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 14:50:04.103612 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4e5cc5-f2b1-4081-9c21-1834c64426d9 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:50:04.114193 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4e5cc5-f2b1-4081-9c21-1834c64426d9 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:50:04.118796 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c99b3d-90da-4e38-8ffa-18914bd6a5af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:50:04.124395 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c99b3d-90da-4e38-8ffa-18914bd6a5af admin admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:04.124395 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c99b3d-90da-4e38-8ffa-18914bd6a5af admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:04.124395 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c99b3d-90da-4e38-8ffa-18914bd6a5af admin admin] PATH_INFO: `/v3/projects/eda5305e9a18414d8af4b53160bfbc01/users/ac1fbbe22fd84950b2ce26518c2750ca/roles/9baff2b18f2649f7b757ff2e1f63c853` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:04.135742 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c99b3d-90da-4e38-8ffa-18914bd6a5af admin admin] RBAC: Authorizing `identity:create_grant(project_id=eda5305e9a18414d8af4b53160bfbc01, user_id=ac1fbbe22fd84950b2ce26518c2750ca, role_id=9baff2b18f2649f7b757ff2e1f63c853)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:50:04.138528 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c99b3d-90da-4e38-8ffa-18914bd6a5af admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:50:04.207664 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1274/2515] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:50:04 2022] PUT /identity/v3/projects/eda5305e9a18414d8af4b53160bfbc01/users/ac1fbbe22fd84950b2ce26518c2750ca/roles/9baff2b18f2649f7b757ff2e1f63c853 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:50:04.213646 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2130ad00-5373-4e28-a441-dedf237af2b8 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:50:04.230860 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2130ad00-5373-4e28-a441-dedf237af2b8 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:50:04.293480 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0a38b6-7bea-40f5-a845-92a6ea80444a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:50:04.294143 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0a38b6-7bea-40f5-a845-92a6ea80444a admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:04.296442 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0a38b6-7bea-40f5-a845-92a6ea80444a admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:04.296442 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0a38b6-7bea-40f5-a845-92a6ea80444a admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:04.296442 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0a38b6-7bea-40f5-a845-92a6ea80444a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:50:04.297202 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0a38b6-7bea-40f5-a845-92a6ea80444a admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:50:04.306716 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1242/2516] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:50:04 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:50:04.310900 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c99b3d-90da-4e38-8ffa-18914bd6a5af admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:50:04.321045 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c99b3d-90da-4e38-8ffa-18914bd6a5af admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:50:04.326133 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce8d0b7-7bc3-44a4-8a12-ab37c45a44ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:50:04.327000 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce8d0b7-7bc3-44a4-8a12-ab37c45a44ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:04.330474 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce8d0b7-7bc3-44a4-8a12-ab37c45a44ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:04.330474 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce8d0b7-7bc3-44a4-8a12-ab37c45a44ba admin admin] PATH_INFO: `/v3/projects/eda5305e9a18414d8af4b53160bfbc01/users/ac1fbbe22fd84950b2ce26518c2750ca/roles/9baff2b18f2649f7b757ff2e1f63c853` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:04.330474 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce8d0b7-7bc3-44a4-8a12-ab37c45a44ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=eda5305e9a18414d8af4b53160bfbc01, user_id=ac1fbbe22fd84950b2ce26518c2750ca, role_id=9baff2b18f2649f7b757ff2e1f63c853)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:50:04.332469 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce8d0b7-7bc3-44a4-8a12-ab37c45a44ba admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:50:04.343327 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1275/2517] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:50:04 2022] PUT /identity/v3/projects/eda5305e9a18414d8af4b53160bfbc01/users/ac1fbbe22fd84950b2ce26518c2750ca/roles/9baff2b18f2649f7b757ff2e1f63c853 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:50:04.354718 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3514fa0-63f4-437a-8b73-fdc3dfb875af None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:04.355012 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3514fa0-63f4-437a-8b73-fdc3dfb875af None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:04.355295 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3514fa0-63f4-437a-8b73-fdc3dfb875af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:04.381225 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-a3514fa0-63f4-437a-8b73-fdc3dfb875af None None] MFA Rules not processed for user `ac1fbbe22fd84950b2ce26518c2750ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:50:04.382429 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-a3514fa0-63f4-437a-8b73-fdc3dfb875af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:50:04.430255 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:50:04.451529 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1243/2518] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:50:04 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:50:04.463405 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce8d0b7-7bc3-44a4-8a12-ab37c45a44ba admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:50:04.465620 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-3ce8d0b7-7bc3-44a4-8a12-ab37c45a44ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:50:04.526268 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce8d0b7-7bc3-44a4-8a12-ab37c45a44ba admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:50:04.530659 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c01139-4c40-4dd4-9270-35655e5f7735 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d81f597cb44c56a966537089bcce4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:50:04.531570 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c01139-4c40-4dd4-9270-35655e5f7735 service cinder] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:04.531948 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c01139-4c40-4dd4-9270-35655e5f7735 service cinder] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:04.532246 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c01139-4c40-4dd4-9270-35655e5f7735 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:04.593972 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c01139-4c40-4dd4-9270-35655e5f7735 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:50:04.595588 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c01139-4c40-4dd4-9270-35655e5f7735 service cinder] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:50:04.604943 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1276/2519] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:50:04 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:50:12.850200 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3514fa0-63f4-437a-8b73-fdc3dfb875af None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:50:12.851657 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-a3514fa0-63f4-437a-8b73-fdc3dfb875af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:50:12.863742 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3514fa0-63f4-437a-8b73-fdc3dfb875af None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:50:12.867941 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d62727-2cf4-451e-9cb2-e51d062905f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d81f597cb44c56a966537089bcce4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:50:12.868823 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d62727-2cf4-451e-9cb2-e51d062905f9 service cinder] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:12.869153 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d62727-2cf4-451e-9cb2-e51d062905f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:12.869530 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d62727-2cf4-451e-9cb2-e51d062905f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:12.879848 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d62727-2cf4-451e-9cb2-e51d062905f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:50:12.881316 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d62727-2cf4-451e-9cb2-e51d062905f9 service cinder] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:50:12.892606 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1244/2520] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:50:12 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:50:13.323802 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c01139-4c40-4dd4-9270-35655e5f7735 service cinder] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:50:13.325982 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-20c01139-4c40-4dd4-9270-35655e5f7735 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:50:13.383196 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:50:13.401397 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c01139-4c40-4dd4-9270-35655e5f7735 service cinder] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:50:13.407052 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebff0ec9-bc05-4629-8a43-d959d2343a25 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e857db4902e42989d0d040bbc8517c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:50:13.407711 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebff0ec9-bc05-4629-8a43-d959d2343a25 service nova] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:13.407998 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebff0ec9-bc05-4629-8a43-d959d2343a25 service nova] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:13.408276 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebff0ec9-bc05-4629-8a43-d959d2343a25 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:13.410098 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-ebff0ec9-bc05-4629-8a43-d959d2343a25 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:50:13.425451 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebff0ec9-bc05-4629-8a43-d959d2343a25 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:50:13.427048 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebff0ec9-bc05-4629-8a43-d959d2343a25 service nova] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:50:13.438635 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1277/2521] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:50:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:50:21.732695 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d62727-2cf4-451e-9cb2-e51d062905f9 service cinder] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:50:21.734198 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-c7d62727-2cf4-451e-9cb2-e51d062905f9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:50:21.799317 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d62727-2cf4-451e-9cb2-e51d062905f9 service cinder] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:50:21.803317 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db121f2d-ca20-448b-a811-ef13e9d6c6d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:50:21.803820 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db121f2d-ca20-448b-a811-ef13e9d6c6d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:21.807921 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db121f2d-ca20-448b-a811-ef13e9d6c6d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:21.807921 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db121f2d-ca20-448b-a811-ef13e9d6c6d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:21.807921 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-db121f2d-ca20-448b-a811-ef13e9d6c6d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:50:21.872722 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db121f2d-ca20-448b-a811-ef13e9d6c6d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:50:21.874164 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db121f2d-ca20-448b-a811-ef13e9d6c6d6 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:50:21.883974 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1245/2522] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:50:21 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:50:23.491978 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=8 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:50:31.879412 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=62 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:50:41.334546 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebff0ec9-bc05-4629-8a43-d959d2343a25 service nova] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:50:41.336444 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-ebff0ec9-bc05-4629-8a43-d959d2343a25 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:50:41.349301 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebff0ec9-bc05-4629-8a43-d959d2343a25 service nova] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:50:41.353764 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f26d3a8-d912-49c0-bae0-3c048e7cb93f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:50:41.354464 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f26d3a8-d912-49c0-bae0-3c048e7cb93f service neutron] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:41.354773 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f26d3a8-d912-49c0-bae0-3c048e7cb93f service neutron] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:41.355058 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f26d3a8-d912-49c0-bae0-3c048e7cb93f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:41.357051 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-8f26d3a8-d912-49c0-bae0-3c048e7cb93f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:50:41.426579 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f26d3a8-d912-49c0-bae0-3c048e7cb93f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:50:41.429413 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f26d3a8-d912-49c0-bae0-3c048e7cb93f service neutron] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:50:41.441993 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1278/2523] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:50:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 14:50:45.219308 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db121f2d-ca20-448b-a811-ef13e9d6c6d6 service neutron] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:50:45.220991 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-db121f2d-ca20-448b-a811-ef13e9d6c6d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:50:45.237185 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db121f2d-ca20-448b-a811-ef13e9d6c6d6 service neutron] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:50:45.242369 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c5b269-3278-4172-b282-fcec11b7e6bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:50:45.243243 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c5b269-3278-4172-b282-fcec11b7e6bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:45.243521 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c5b269-3278-4172-b282-fcec11b7e6bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:45.243790 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c5b269-3278-4172-b282-fcec11b7e6bc admin admin] PATH_INFO: `/v3/users/34eb9ec1f3ff44bbb03cf934cb2a04de` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:45.259150 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c5b269-3278-4172-b282-fcec11b7e6bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=34eb9ec1f3ff44bbb03cf934cb2a04de)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:50:45.261103 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c5b269-3278-4172-b282-fcec11b7e6bc admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:50:45.332975 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f26d3a8-d912-49c0-bae0-3c048e7cb93f service neutron] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:50:45.343983 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f26d3a8-d912-49c0-bae0-3c048e7cb93f service neutron] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:50:45.348628 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7044a9a-d4b3-4ef6-93a6-05b70b1695b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:50:45.350398 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7044a9a-d4b3-4ef6-93a6-05b70b1695b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:45.350641 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7044a9a-d4b3-4ef6-93a6-05b70b1695b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:45.350913 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7044a9a-d4b3-4ef6-93a6-05b70b1695b9 admin admin] PATH_INFO: `/v3/users/d0d18d877bf84ba3a32ca0ce13f89816` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:45.352282 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7044a9a-d4b3-4ef6-93a6-05b70b1695b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0d18d877bf84ba3a32ca0ce13f89816)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:50:45.356578 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7044a9a-d4b3-4ef6-93a6-05b70b1695b9 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:50:45.473191 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-94c5b269-3278-4172-b282-fcec11b7e6bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34eb9ec1f3ff44bbb03cf934cb2a04de'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:50:45.474131 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-94c5b269-3278-4172-b282-fcec11b7e6bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34eb9ec1f3ff44bbb03cf934cb2a04de'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:50:45.475582 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-94c5b269-3278-4172-b282-fcec11b7e6bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34eb9ec1f3ff44bbb03cf934cb2a04de'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:50:45.483045 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-94c5b269-3278-4172-b282-fcec11b7e6bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34eb9ec1f3ff44bbb03cf934cb2a04de'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:50:45.496297 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-94c5b269-3278-4172-b282-fcec11b7e6bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34eb9ec1f3ff44bbb03cf934cb2a04de'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:50:45.551719 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-e7044a9a-d4b3-4ef6-93a6-05b70b1695b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0d18d877bf84ba3a32ca0ce13f89816'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:50:45.552357 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-e7044a9a-d4b3-4ef6-93a6-05b70b1695b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0d18d877bf84ba3a32ca0ce13f89816'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:50:45.553129 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-e7044a9a-d4b3-4ef6-93a6-05b70b1695b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0d18d877bf84ba3a32ca0ce13f89816'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:50:45.559973 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-e7044a9a-d4b3-4ef6-93a6-05b70b1695b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0d18d877bf84ba3a32ca0ce13f89816'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:50:45.568596 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1246/2524] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:50:45 2022] DELETE /identity/v3/users/34eb9ec1f3ff44bbb03cf934cb2a04de => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:50:45.574025 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-e7044a9a-d4b3-4ef6-93a6-05b70b1695b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0d18d877bf84ba3a32ca0ce13f89816'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:50:45.668514 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1279/2525] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:50:45 2022] DELETE /identity/v3/users/d0d18d877bf84ba3a32ca0ce13f89816 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:50:45.673074 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca939766-4e35-471f-91fd-52282846a732 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:45.673368 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca939766-4e35-471f-91fd-52282846a732 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:45.673681 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca939766-4e35-471f-91fd-52282846a732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:45.691619 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-ca939766-4e35-471f-91fd-52282846a732 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:50:45.692923 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-ca939766-4e35-471f-91fd-52282846a732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:50:45.753329 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1247/2526] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:50:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:50:45.763563 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7044a9a-d4b3-4ef6-93a6-05b70b1695b9 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:50:45.765020 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-e7044a9a-d4b3-4ef6-93a6-05b70b1695b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:50:45.813458 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca939766-4e35-471f-91fd-52282846a732 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:50:45.814889 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-ca939766-4e35-471f-91fd-52282846a732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:50:45.825010 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7044a9a-d4b3-4ef6-93a6-05b70b1695b9 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:50:45.827730 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca939766-4e35-471f-91fd-52282846a732 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:50:45.829296 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fc9502-8fad-41dd-9ca0-055fa0d5a821 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:50:45.830017 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fc9502-8fad-41dd-9ca0-055fa0d5a821 service neutron] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:45.830309 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fc9502-8fad-41dd-9ca0-055fa0d5a821 service neutron] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:45.830591 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fc9502-8fad-41dd-9ca0-055fa0d5a821 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:45.832705 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ccbe5a-7bd1-4285-b942-03a8c8f0eefa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:50:45.833606 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ccbe5a-7bd1-4285-b942-03a8c8f0eefa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:45.834467 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ccbe5a-7bd1-4285-b942-03a8c8f0eefa admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:45.834754 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ccbe5a-7bd1-4285-b942-03a8c8f0eefa admin admin] PATH_INFO: `/v3/projects/8eae64ee53144e4f8d583bb4288c0aeb` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:45.842305 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fc9502-8fad-41dd-9ca0-055fa0d5a821 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:50:45.844035 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fc9502-8fad-41dd-9ca0-055fa0d5a821 service neutron] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:50:45.853369 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ccbe5a-7bd1-4285-b942-03a8c8f0eefa admin admin] RBAC: Authorizing `identity:delete_project(project_id=8eae64ee53144e4f8d583bb4288c0aeb)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:50:45.854865 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1280/2527] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:50:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:50:45.855339 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ccbe5a-7bd1-4285-b942-03a8c8f0eefa admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:50:45.973043 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:50:45.975198 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a6ccbe5a-7bd1-4285-b942-03a8c8f0eefa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8eae64ee53144e4f8d583bb4288c0aeb', 'request_id': 'req-a6ccbe5a-7bd1-4285-b942-03a8c8f0eefa'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:50:46.053277 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a6ccbe5a-7bd1-4285-b942-03a8c8f0eefa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8eae64ee53144e4f8d583bb4288c0aeb', 'request_id': 'req-a6ccbe5a-7bd1-4285-b942-03a8c8f0eefa'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:50:46.062570 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a6ccbe5a-7bd1-4285-b942-03a8c8f0eefa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8eae64ee53144e4f8d583bb4288c0aeb', 'request_id': 'req-a6ccbe5a-7bd1-4285-b942-03a8c8f0eefa'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:50:46.090278 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a6ccbe5a-7bd1-4285-b942-03a8c8f0eefa admin admin] The token cache is being invalidate because project 8eae64ee53144e4f8d583bb4288c0aeb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:50:46.090278 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a6ccbe5a-7bd1-4285-b942-03a8c8f0eefa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:50:46.090278 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a6ccbe5a-7bd1-4285-b942-03a8c8f0eefa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:50:46.090278 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1248/2528] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:50:45 2022] DELETE /identity/v3/projects/8eae64ee53144e4f8d583bb4288c0aeb => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:50:46.095178 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5788ea0a-c7c5-4746-8552-549d3b586f94 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:46.095560 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5788ea0a-c7c5-4746-8552-549d3b586f94 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:46.096051 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5788ea0a-c7c5-4746-8552-549d3b586f94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:46.117390 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-5788ea0a-c7c5-4746-8552-549d3b586f94 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:50:46.121706 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-5788ea0a-c7c5-4746-8552-549d3b586f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:50:46.123141 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:50:46.184756 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1281/2529] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:50:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:50:46.194157 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9609fb-87b9-4668-8a02-5e50d9d19da3 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:46.199123 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9609fb-87b9-4668-8a02-5e50d9d19da3 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:46.199123 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9609fb-87b9-4668-8a02-5e50d9d19da3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:46.213198 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-0d9609fb-87b9-4668-8a02-5e50d9d19da3 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:50:46.214489 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-0d9609fb-87b9-4668-8a02-5e50d9d19da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:50:46.222545 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1249/2530] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:50:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:50:46.233584 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5788ea0a-c7c5-4746-8552-549d3b586f94 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:50:46.236583 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5788ea0a-c7c5-4746-8552-549d3b586f94 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:50:46.241805 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972e12d3-9dc4-4653-820a-4511a71bd1be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:50:46.242464 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972e12d3-9dc4-4653-820a-4511a71bd1be admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:46.242746 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972e12d3-9dc4-4653-820a-4511a71bd1be admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:46.243021 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972e12d3-9dc4-4653-820a-4511a71bd1be admin admin] PATH_INFO: `/v3/domains` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:46.243750 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972e12d3-9dc4-4653-820a-4511a71bd1be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:50:46.244140 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972e12d3-9dc4-4653-820a-4511a71bd1be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:50:46.245786 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972e12d3-9dc4-4653-820a-4511a71bd1be admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:50:46.257982 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1282/2531] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jun 7 14:50:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:50:46.262567 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9609fb-87b9-4668-8a02-5e50d9d19da3 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:50:46.272440 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9609fb-87b9-4668-8a02-5e50d9d19da3 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:50:46.277464 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7492c413-64d9-4927-8e77-8699606a0dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:50:46.278116 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7492c413-64d9-4927-8e77-8699606a0dcc admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:46.278411 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7492c413-64d9-4927-8e77-8699606a0dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:46.278692 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7492c413-64d9-4927-8e77-8699606a0dcc admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:46.279484 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7492c413-64d9-4927-8e77-8699606a0dcc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:50:46.281453 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7492c413-64d9-4927-8e77-8699606a0dcc admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:50:46.357804 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972e12d3-9dc4-4653-820a-4511a71bd1be admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:50:46.360675 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-972e12d3-9dc4-4653-820a-4511a71bd1be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:50:46.374080 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972e12d3-9dc4-4653-820a-4511a71bd1be admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:50:46.380059 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f4f78d-ed65-4275-951c-b6899d760d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:50:46.385756 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f4f78d-ed65-4275-951c-b6899d760d33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:46.385756 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f4f78d-ed65-4275-951c-b6899d760d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:46.385756 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f4f78d-ed65-4275-951c-b6899d760d33 admin admin] PATH_INFO: `/v3/projects/5134adfcb5844a02a6d9d46ba2113147` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:46.385756 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f4f78d-ed65-4275-951c-b6899d760d33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5134adfcb5844a02a6d9d46ba2113147)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:50:46.385756 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f4f78d-ed65-4275-951c-b6899d760d33 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:50:46.390263 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1250/2532] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:50:46 2022] POST /identity/v3/projects => generated 381 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:50:46.391163 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7492c413-64d9-4927-8e77-8699606a0dcc admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:50:46.455645 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7492c413-64d9-4927-8e77-8699606a0dcc admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:50:46.460864 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2516483f-b187-4546-801c-b6fa66d775e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:50:46.461576 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2516483f-b187-4546-801c-b6fa66d775e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:46.461931 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2516483f-b187-4546-801c-b6fa66d775e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:46.462282 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2516483f-b187-4546-801c-b6fa66d775e6 admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:46.462965 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2516483f-b187-4546-801c-b6fa66d775e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:50:46.464911 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2516483f-b187-4546-801c-b6fa66d775e6 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:50:46.510029 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-49f4f78d-ed65-4275-951c-b6899d760d33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5134adfcb5844a02a6d9d46ba2113147', 'request_id': 'req-49f4f78d-ed65-4275-951c-b6899d760d33'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:50:46.555373 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1251/2533] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:50:46 2022] POST /identity/v3/users => generated 291 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:50:46.559505 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2516483f-b187-4546-801c-b6fa66d775e6 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:50:46.569462 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2516483f-b187-4546-801c-b6fa66d775e6 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:50:46.572505 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-49f4f78d-ed65-4275-951c-b6899d760d33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5134adfcb5844a02a6d9d46ba2113147', 'request_id': 'req-49f4f78d-ed65-4275-951c-b6899d760d33'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:50:46.578892 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-49f4f78d-ed65-4275-951c-b6899d760d33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5134adfcb5844a02a6d9d46ba2113147', 'request_id': 'req-49f4f78d-ed65-4275-951c-b6899d760d33'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:50:46.592650 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-49f4f78d-ed65-4275-951c-b6899d760d33 admin admin] The token cache is being invalidate because project 5134adfcb5844a02a6d9d46ba2113147 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:50:46.592944 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-49f4f78d-ed65-4275-951c-b6899d760d33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:50:46.593650 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-49f4f78d-ed65-4275-951c-b6899d760d33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:50:46.595670 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1283/2534] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:50:46 2022] DELETE /identity/v3/projects/5134adfcb5844a02a6d9d46ba2113147 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:50:46.599823 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f4f78d-ed65-4275-951c-b6899d760d33 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:50:46.604799 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-49f4f78d-ed65-4275-951c-b6899d760d33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:50:46.627907 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada4eaae-5faf-4ceb-ae21-6082158e6196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:50:46.628661 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada4eaae-5faf-4ceb-ae21-6082158e6196 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:46.628928 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada4eaae-5faf-4ceb-ae21-6082158e6196 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:46.629214 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada4eaae-5faf-4ceb-ae21-6082158e6196 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:46.629972 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada4eaae-5faf-4ceb-ae21-6082158e6196 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:50:46.632145 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada4eaae-5faf-4ceb-ae21-6082158e6196 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:50:46.642578 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1252/2535] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:50:46 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 14:50:46.647157 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada4eaae-5faf-4ceb-ae21-6082158e6196 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:50:46.649435 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-ada4eaae-5faf-4ceb-ae21-6082158e6196 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:50:46.662437 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada4eaae-5faf-4ceb-ae21-6082158e6196 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:50:46.667298 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d1194b-5f4d-430e-94b9-ac776094ee3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:50:46.667967 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d1194b-5f4d-430e-94b9-ac776094ee3c admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:46.668074 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f4f78d-ed65-4275-951c-b6899d760d33 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:50:46.668290 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d1194b-5f4d-430e-94b9-ac776094ee3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:46.668618 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d1194b-5f4d-430e-94b9-ac776094ee3c admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:46.669335 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d1194b-5f4d-430e-94b9-ac776094ee3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:50:46.671191 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d1194b-5f4d-430e-94b9-ac776094ee3c admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:50:46.672916 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba006e92-3333-4f8f-8b0f-9deff2dbcb2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:50:46.673782 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba006e92-3333-4f8f-8b0f-9deff2dbcb2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:46.674065 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba006e92-3333-4f8f-8b0f-9deff2dbcb2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:46.674350 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba006e92-3333-4f8f-8b0f-9deff2dbcb2f admin admin] PATH_INFO: `/v3/users/ac1fbbe22fd84950b2ce26518c2750ca` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:46.675562 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba006e92-3333-4f8f-8b0f-9deff2dbcb2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac1fbbe22fd84950b2ce26518c2750ca)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:50:46.677634 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba006e92-3333-4f8f-8b0f-9deff2dbcb2f admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:50:46.681444 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1253/2536] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:50:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 14:50:46.685617 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d1194b-5f4d-430e-94b9-ac776094ee3c admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:50:46.696857 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d1194b-5f4d-430e-94b9-ac776094ee3c admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:50:46.700503 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:50:46.701755 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8add476-a9c7-41c8-b2e1-3010ea690dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:50:46.702669 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8add476-a9c7-41c8-b2e1-3010ea690dd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:46.702956 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8add476-a9c7-41c8-b2e1-3010ea690dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:46.703237 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8add476-a9c7-41c8-b2e1-3010ea690dd4 admin admin] PATH_INFO: `/v3/projects/b803381e2ff74a1883ce59917d7e70ea/users/5f4058d42c0e467f81d709ac42f7ef94/roles/3771341b3be641139e752aab690397a4` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:46.719053 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8add476-a9c7-41c8-b2e1-3010ea690dd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b803381e2ff74a1883ce59917d7e70ea, user_id=5f4058d42c0e467f81d709ac42f7ef94, role_id=3771341b3be641139e752aab690397a4)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:50:46.721713 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8add476-a9c7-41c8-b2e1-3010ea690dd4 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:50:46.830494 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=91,keystone:INSERT=6,keystone:DELETE=3 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:50:46.836395 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1254/2537] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:50:46 2022] PUT /identity/v3/projects/b803381e2ff74a1883ce59917d7e70ea/users/5f4058d42c0e467f81d709ac42f7ef94/roles/3771341b3be641139e752aab690397a4 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:50:46.846435 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cd373b-da7b-407f-9e5e-f4c558d7ed96 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:46.846729 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cd373b-da7b-407f-9e5e-f4c558d7ed96 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:46.847056 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cd373b-da7b-407f-9e5e-f4c558d7ed96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:46.874142 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-48cd373b-da7b-407f-9e5e-f4c558d7ed96 None None] MFA Rules not processed for user `5f4058d42c0e467f81d709ac42f7ef94`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:50:46.875258 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-48cd373b-da7b-407f-9e5e-f4c558d7ed96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:50:46.899777 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-ba006e92-3333-4f8f-8b0f-9deff2dbcb2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac1fbbe22fd84950b2ce26518c2750ca'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:50:46.900520 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-ba006e92-3333-4f8f-8b0f-9deff2dbcb2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac1fbbe22fd84950b2ce26518c2750ca'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:50:46.901387 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-ba006e92-3333-4f8f-8b0f-9deff2dbcb2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac1fbbe22fd84950b2ce26518c2750ca'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:50:46.910931 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-ba006e92-3333-4f8f-8b0f-9deff2dbcb2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac1fbbe22fd84950b2ce26518c2750ca'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:50:46.922999 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-ba006e92-3333-4f8f-8b0f-9deff2dbcb2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac1fbbe22fd84950b2ce26518c2750ca'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:50:46.941058 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1255/2538] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:50:46 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:50:46.946229 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cd373b-da7b-407f-9e5e-f4c558d7ed96 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:50:46.947539 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-48cd373b-da7b-407f-9e5e-f4c558d7ed96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:50:46.988977 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1284/2539] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:50:46 2022] DELETE /identity/v3/users/ac1fbbe22fd84950b2ce26518c2750ca => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:50:47.013759 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cd373b-da7b-407f-9e5e-f4c558d7ed96 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:50:47.075344 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18be574-03ea-4089-a05b-3d25e17961b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:50:47.076018 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18be574-03ea-4089-a05b-3d25e17961b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:47.076306 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18be574-03ea-4089-a05b-3d25e17961b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:47.076610 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18be574-03ea-4089-a05b-3d25e17961b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:47.077307 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18be574-03ea-4089-a05b-3d25e17961b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:50:47.079280 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18be574-03ea-4089-a05b-3d25e17961b7 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:50:47.208235 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1256/2540] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:50:46 2022] POST /identity/v3/projects => generated 381 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:50:47.212449 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba006e92-3333-4f8f-8b0f-9deff2dbcb2f admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:50:47.225128 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba006e92-3333-4f8f-8b0f-9deff2dbcb2f admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:50:47.288399 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e01e56-657b-4891-83be-82f1c9c4e0da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:50:47.290092 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e01e56-657b-4891-83be-82f1c9c4e0da admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:47.290092 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e01e56-657b-4891-83be-82f1c9c4e0da admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:47.290092 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e01e56-657b-4891-83be-82f1c9c4e0da admin admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:47.290659 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e01e56-657b-4891-83be-82f1c9c4e0da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:50:47.295813 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e01e56-657b-4891-83be-82f1c9c4e0da admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:50:47.412897 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1285/2541] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:50:47 2022] POST /identity/v3/users => generated 297 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:50:47.416917 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18be574-03ea-4089-a05b-3d25e17961b7 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:50:47.427089 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18be574-03ea-4089-a05b-3d25e17961b7 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:50:47.433592 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb7731d-e40d-48df-b53f-cb3c64d898f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:50:47.434267 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb7731d-e40d-48df-b53f-cb3c64d898f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:47.434555 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb7731d-e40d-48df-b53f-cb3c64d898f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:47.434838 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb7731d-e40d-48df-b53f-cb3c64d898f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:47.435579 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb7731d-e40d-48df-b53f-cb3c64d898f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:50:47.437210 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb7731d-e40d-48df-b53f-cb3c64d898f3 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:50:47.448045 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e01e56-657b-4891-83be-82f1c9c4e0da admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:50:47.450150 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-54e01e56-657b-4891-83be-82f1c9c4e0da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:50:47.450291 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1257/2542] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:50:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:50:47.456771 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb7731d-e40d-48df-b53f-cb3c64d898f3 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:50:47.464506 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e01e56-657b-4891-83be-82f1c9c4e0da admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:50:47.470284 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb7731d-e40d-48df-b53f-cb3c64d898f3 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:50:47.473714 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cd8210-2200-4cf5-b7bd-e7ecc4b579c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:50:47.473714 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cd8210-2200-4cf5-b7bd-e7ecc4b579c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:47.473714 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cd8210-2200-4cf5-b7bd-e7ecc4b579c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:47.473714 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cd8210-2200-4cf5-b7bd-e7ecc4b579c8 admin admin] PATH_INFO: `/v3/projects/eda5305e9a18414d8af4b53160bfbc01` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:47.474539 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cd8210-2200-4cf5-b7bd-e7ecc4b579c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eda5305e9a18414d8af4b53160bfbc01)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:50:47.476504 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cd8210-2200-4cf5-b7bd-e7ecc4b579c8 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:50:47.477406 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21154c3-db2e-49d8-9af5-0e0e5a91790d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:50:47.478761 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21154c3-db2e-49d8-9af5-0e0e5a91790d admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:47.479126 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21154c3-db2e-49d8-9af5-0e0e5a91790d admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:47.479387 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21154c3-db2e-49d8-9af5-0e0e5a91790d admin admin] PATH_INFO: `/v3/projects/d008d0de59064c14b83275efaadb3c0e/users/ace643dc2641461ead9b135ba23087aa/roles/9baff2b18f2649f7b757ff2e1f63c853` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:47.483303 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:50:47.497227 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21154c3-db2e-49d8-9af5-0e0e5a91790d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d008d0de59064c14b83275efaadb3c0e, user_id=ace643dc2641461ead9b135ba23087aa, role_id=9baff2b18f2649f7b757ff2e1f63c853)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:50:47.499895 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21154c3-db2e-49d8-9af5-0e0e5a91790d admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:50:47.613162 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1258/2543] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:50:47 2022] PUT /identity/v3/projects/d008d0de59064c14b83275efaadb3c0e/users/ace643dc2641461ead9b135ba23087aa/roles/9baff2b18f2649f7b757ff2e1f63c853 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:50:47.617765 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21154c3-db2e-49d8-9af5-0e0e5a91790d admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:50:47.642400 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-70cd8210-2200-4cf5-b7bd-e7ecc4b579c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eda5305e9a18414d8af4b53160bfbc01', 'request_id': 'req-70cd8210-2200-4cf5-b7bd-e7ecc4b579c8'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:50:47.680103 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21154c3-db2e-49d8-9af5-0e0e5a91790d admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:50:47.692423 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-70cd8210-2200-4cf5-b7bd-e7ecc4b579c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eda5305e9a18414d8af4b53160bfbc01', 'request_id': 'req-70cd8210-2200-4cf5-b7bd-e7ecc4b579c8'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:50:47.699634 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-70cd8210-2200-4cf5-b7bd-e7ecc4b579c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eda5305e9a18414d8af4b53160bfbc01', 'request_id': 'req-70cd8210-2200-4cf5-b7bd-e7ecc4b579c8'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:50:47.711875 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-70cd8210-2200-4cf5-b7bd-e7ecc4b579c8 admin admin] The token cache is being invalidate because project eda5305e9a18414d8af4b53160bfbc01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:50:47.712160 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-70cd8210-2200-4cf5-b7bd-e7ecc4b579c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:50:47.712995 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-70cd8210-2200-4cf5-b7bd-e7ecc4b579c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:50:47.714677 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1286/2544] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:50:47 2022] DELETE /identity/v3/projects/eda5305e9a18414d8af4b53160bfbc01 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:50:47.719587 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29000b8c-e41e-4623-9bcc-ab5b015a5de9 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:47.719946 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29000b8c-e41e-4623-9bcc-ab5b015a5de9 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:47.720306 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29000b8c-e41e-4623-9bcc-ab5b015a5de9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:47.737784 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a40958d-f5a3-4b00-b322-682d382fc810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:50:47.738471 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a40958d-f5a3-4b00-b322-682d382fc810 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:47.744473 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a40958d-f5a3-4b00-b322-682d382fc810 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:47.744473 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a40958d-f5a3-4b00-b322-682d382fc810 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:47.744680 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-29000b8c-e41e-4623-9bcc-ab5b015a5de9 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:50:47.744817 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a40958d-f5a3-4b00-b322-682d382fc810 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:50:47.744919 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-29000b8c-e41e-4623-9bcc-ab5b015a5de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:50:47.745024 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a40958d-f5a3-4b00-b322-682d382fc810 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:50:47.749769 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1287/2545] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:50:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:50:47.752443 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1259/2546] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:50:47 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 14:50:47.756948 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29000b8c-e41e-4623-9bcc-ab5b015a5de9 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:50:47.757843 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1863e90-380f-433f-99be-d4a9aea8a4b0 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:47.758182 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1863e90-380f-433f-99be-d4a9aea8a4b0 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:47.758302 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-29000b8c-e41e-4623-9bcc-ab5b015a5de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:50:47.758466 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1863e90-380f-433f-99be-d4a9aea8a4b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:47.767379 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:DELETE=14,keystone:SELECT=81,keystone:INSERT=5 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:50:47.771369 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29000b8c-e41e-4623-9bcc-ab5b015a5de9 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:50:47.776408 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2225240-95bd-4d92-b768-024dca10a3b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:50:47.777418 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2225240-95bd-4d92-b768-024dca10a3b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:47.777756 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2225240-95bd-4d92-b768-024dca10a3b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:47.778033 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2225240-95bd-4d92-b768-024dca10a3b9 admin admin] PATH_INFO: `/v3/projects/d008d0de59064c14b83275efaadb3c0e/users/ace643dc2641461ead9b135ba23087aa/roles/9baff2b18f2649f7b757ff2e1f63c853` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:47.779628 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-d1863e90-380f-433f-99be-d4a9aea8a4b0 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:50:47.779949 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2225240-95bd-4d92-b768-024dca10a3b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d008d0de59064c14b83275efaadb3c0e, user_id=ace643dc2641461ead9b135ba23087aa, role_id=9baff2b18f2649f7b757ff2e1f63c853)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:50:47.781019 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-d1863e90-380f-433f-99be-d4a9aea8a4b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:50:47.782797 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2225240-95bd-4d92-b768-024dca10a3b9 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:50:47.789618 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1260/2547] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:50:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:50:47.793279 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1863e90-380f-433f-99be-d4a9aea8a4b0 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:50:47.796613 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1288/2548] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:50:47 2022] PUT /identity/v3/projects/d008d0de59064c14b83275efaadb3c0e/users/ace643dc2641461ead9b135ba23087aa/roles/9baff2b18f2649f7b757ff2e1f63c853 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:50:47.803430 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1863e90-380f-433f-99be-d4a9aea8a4b0 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:50:47.808742 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c187905e-ee75-4843-96e3-793d8b97413d None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:47.809032 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c187905e-ee75-4843-96e3-793d8b97413d None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:47.809337 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c187905e-ee75-4843-96e3-793d8b97413d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:47.839294 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-c187905e-ee75-4843-96e3-793d8b97413d None None] MFA Rules not processed for user `ace643dc2641461ead9b135ba23087aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:50:47.840524 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-c187905e-ee75-4843-96e3-793d8b97413d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:50:47.863418 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30acca4-ed65-420f-bed5-010c1d58e199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:50:47.864093 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30acca4-ed65-420f-bed5-010c1d58e199 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:47.864409 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30acca4-ed65-420f-bed5-010c1d58e199 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:47.864722 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30acca4-ed65-420f-bed5-010c1d58e199 admin admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:47.865559 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30acca4-ed65-420f-bed5-010c1d58e199 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:50:47.865852 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30acca4-ed65-420f-bed5-010c1d58e199 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:50:47.867711 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30acca4-ed65-420f-bed5-010c1d58e199 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:50:47.880041 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1261/2549] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jun 7 14:50:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:50:47.886168 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30acca4-ed65-420f-bed5-010c1d58e199 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:50:47.895192 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30acca4-ed65-420f-bed5-010c1d58e199 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:50:47.900512 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7cca56-282f-4943-92b2-8b663835475e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:50:47.901430 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7cca56-282f-4943-92b2-8b663835475e admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:47.901843 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7cca56-282f-4943-92b2-8b663835475e admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:47.902133 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7cca56-282f-4943-92b2-8b663835475e admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:47.902840 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7cca56-282f-4943-92b2-8b663835475e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:50:47.904766 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7cca56-282f-4943-92b2-8b663835475e admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:50:47.913582 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1289/2550] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:50:47 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:50:47.919796 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:50:47.925193 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c187905e-ee75-4843-96e3-793d8b97413d None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:50:47.926594 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-c187905e-ee75-4843-96e3-793d8b97413d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:50:47.982395 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1262/2551] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:50:47 2022] POST /identity/v3/projects => generated 379 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:50:47.983909 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7cca56-282f-4943-92b2-8b663835475e admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:50:47.988496 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c187905e-ee75-4843-96e3-793d8b97413d None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:50:48.039243 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d8f1ea-96a0-4a8b-bedc-d0bd6112219c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d81f597cb44c56a966537089bcce4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:50:48.039900 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d8f1ea-96a0-4a8b-bedc-d0bd6112219c service cinder] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:48.040186 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d8f1ea-96a0-4a8b-bedc-d0bd6112219c service cinder] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:48.040495 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d8f1ea-96a0-4a8b-bedc-d0bd6112219c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:48.046544 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7cca56-282f-4943-92b2-8b663835475e admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:50:48.051378 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20070ad2-3789-4712-bbec-8edd212d2579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:50:48.052045 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20070ad2-3789-4712-bbec-8edd212d2579 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:48.052325 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20070ad2-3789-4712-bbec-8edd212d2579 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:48.052665 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20070ad2-3789-4712-bbec-8edd212d2579 admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:48.053354 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20070ad2-3789-4712-bbec-8edd212d2579 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:50:48.055262 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20070ad2-3789-4712-bbec-8edd212d2579 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:50:48.085967 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:50:48.109133 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d8f1ea-96a0-4a8b-bedc-d0bd6112219c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:50:48.110558 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d8f1ea-96a0-4a8b-bedc-d0bd6112219c service cinder] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:50:48.120055 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1290/2552] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:50:47 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 14:50:48.150132 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1263/2553] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:50:47 2022] POST /identity/v3/users => generated 290 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:50:48.153767 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d8f1ea-96a0-4a8b-bedc-d0bd6112219c service cinder] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:50:48.164473 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d8f1ea-96a0-4a8b-bedc-d0bd6112219c service cinder] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:50:48.169821 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dd6b26-18fe-450c-aee8-8cf370ad9ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:50:48.170582 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dd6b26-18fe-450c-aee8-8cf370ad9ca5 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:48.170841 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dd6b26-18fe-450c-aee8-8cf370ad9ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:48.171101 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dd6b26-18fe-450c-aee8-8cf370ad9ca5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:48.171809 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dd6b26-18fe-450c-aee8-8cf370ad9ca5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:50:48.173439 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dd6b26-18fe-450c-aee8-8cf370ad9ca5 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:50:48.184029 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1291/2554] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:50:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:50:48.188351 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20070ad2-3789-4712-bbec-8edd212d2579 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:50:48.199427 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20070ad2-3789-4712-bbec-8edd212d2579 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:50:48.204357 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda3146c-dc68-4db9-b82c-36597001ba0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:50:48.205198 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda3146c-dc68-4db9-b82c-36597001ba0d admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:48.205494 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda3146c-dc68-4db9-b82c-36597001ba0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:48.205776 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda3146c-dc68-4db9-b82c-36597001ba0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:48.206512 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda3146c-dc68-4db9-b82c-36597001ba0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:50:48.208123 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda3146c-dc68-4db9-b82c-36597001ba0d admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:50:48.219559 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1264/2555] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:50:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:50:48.223801 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dd6b26-18fe-450c-aee8-8cf370ad9ca5 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:50:48.234129 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dd6b26-18fe-450c-aee8-8cf370ad9ca5 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:50:48.239373 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3143bc-b6a8-4834-8f91-86ffb4230d79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:50:48.240784 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3143bc-b6a8-4834-8f91-86ffb4230d79 admin admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:48.241062 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3143bc-b6a8-4834-8f91-86ffb4230d79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:48.241334 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3143bc-b6a8-4834-8f91-86ffb4230d79 admin admin] PATH_INFO: `/v3/projects/126a4e5f998d4e588ebb72ad0187ca09/users/85d84f05a1b44adfa4f0afb3f22c8fe4/roles/3771341b3be641139e752aab690397a4` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:48.258109 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3143bc-b6a8-4834-8f91-86ffb4230d79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=126a4e5f998d4e588ebb72ad0187ca09, user_id=85d84f05a1b44adfa4f0afb3f22c8fe4, role_id=3771341b3be641139e752aab690397a4)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:50:48.260799 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3143bc-b6a8-4834-8f91-86ffb4230d79 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:50:48.349841 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1292/2556] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:50:48 2022] PUT /identity/v3/projects/126a4e5f998d4e588ebb72ad0187ca09/users/85d84f05a1b44adfa4f0afb3f22c8fe4/roles/3771341b3be641139e752aab690397a4 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:50:48.356305 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda3146c-dc68-4db9-b82c-36597001ba0d admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:50:48.363678 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ddbc85-640b-4a78-93b1-e33399d38571 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:48.364146 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ddbc85-640b-4a78-93b1-e33399d38571 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:48.364702 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ddbc85-640b-4a78-93b1-e33399d38571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:48.368221 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda3146c-dc68-4db9-b82c-36597001ba0d admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:50:48.397288 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-e7ddbc85-640b-4a78-93b1-e33399d38571 None None] MFA Rules not processed for user `85d84f05a1b44adfa4f0afb3f22c8fe4`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:50:48.398505 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-e7ddbc85-640b-4a78-93b1-e33399d38571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:50:48.420273 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02a1384-830f-46fb-b97a-2d92c29c09a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d81f597cb44c56a966537089bcce4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:50:48.421079 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a1384-830f-46fb-b97a-2d92c29c09a7 service cinder] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:48.421434 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a1384-830f-46fb-b97a-2d92c29c09a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:48.424702 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a1384-830f-46fb-b97a-2d92c29c09a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:48.424702 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-b02a1384-830f-46fb-b97a-2d92c29c09a7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:50:48.466519 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1293/2557] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:50:48 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:50:48.471553 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ddbc85-640b-4a78-93b1-e33399d38571 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:50:48.485911 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02a1384-830f-46fb-b97a-2d92c29c09a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:50:48.487350 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02a1384-830f-46fb-b97a-2d92c29c09a7 service cinder] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:50:48.497858 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1265/2558] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:50:48 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:50:48.539992 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ddbc85-640b-4a78-93b1-e33399d38571 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:50:48.544811 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44a6c3a-f61b-4008-9265-9d5508dbb5a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:50:48.545474 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44a6c3a-f61b-4008-9265-9d5508dbb5a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:48.545763 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44a6c3a-f61b-4008-9265-9d5508dbb5a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:48.546046 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44a6c3a-f61b-4008-9265-9d5508dbb5a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:48.546740 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44a6c3a-f61b-4008-9265-9d5508dbb5a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:50:48.548719 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44a6c3a-f61b-4008-9265-9d5508dbb5a7 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:50:48.665721 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1294/2559] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:50:48 2022] POST /identity/v3/projects => generated 377 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:50:48.669800 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02a1384-830f-46fb-b97a-2d92c29c09a7 service cinder] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:50:48.681607 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02a1384-830f-46fb-b97a-2d92c29c09a7 service cinder] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:50:48.705698 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:50:48.738006 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446715a5-0297-44d5-9271-617828804a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:50:48.738688 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446715a5-0297-44d5-9271-617828804a36 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:48.738983 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446715a5-0297-44d5-9271-617828804a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:48.739265 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446715a5-0297-44d5-9271-617828804a36 admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:48.739976 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446715a5-0297-44d5-9271-617828804a36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:50:48.742625 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446715a5-0297-44d5-9271-617828804a36 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:50:48.842158 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1266/2560] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:50:48 2022] POST /identity/v3/users => generated 295 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:50:48.845537 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44a6c3a-f61b-4008-9265-9d5508dbb5a7 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:50:48.856322 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44a6c3a-f61b-4008-9265-9d5508dbb5a7 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:50:48.860968 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d627d531-c7df-469f-93e7-4b945ec863de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:50:48.861677 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d627d531-c7df-469f-93e7-4b945ec863de admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:48.861964 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d627d531-c7df-469f-93e7-4b945ec863de admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:48.862241 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d627d531-c7df-469f-93e7-4b945ec863de admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:48.862968 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d627d531-c7df-469f-93e7-4b945ec863de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:50:48.864610 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d627d531-c7df-469f-93e7-4b945ec863de admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:50:48.876672 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1295/2561] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:50:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:50:48.880893 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446715a5-0297-44d5-9271-617828804a36 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:50:48.891539 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446715a5-0297-44d5-9271-617828804a36 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:50:48.898864 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f222f9d-192b-4ae1-8e87-1bde5e53cd1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:50:48.898864 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f222f9d-192b-4ae1-8e87-1bde5e53cd1d admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:48.898864 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f222f9d-192b-4ae1-8e87-1bde5e53cd1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:48.898864 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f222f9d-192b-4ae1-8e87-1bde5e53cd1d admin admin] PATH_INFO: `/v3/projects/021a562997dc4bec8bda3c9630e8616a/users/1801fc6e799747338a2cd99c300c8405/roles/9baff2b18f2649f7b757ff2e1f63c853` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:48.924980 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f222f9d-192b-4ae1-8e87-1bde5e53cd1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=021a562997dc4bec8bda3c9630e8616a, user_id=1801fc6e799747338a2cd99c300c8405, role_id=9baff2b18f2649f7b757ff2e1f63c853)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:50:48.928095 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f222f9d-192b-4ae1-8e87-1bde5e53cd1d admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:50:48.996340 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1267/2562] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:50:48 2022] PUT /identity/v3/projects/021a562997dc4bec8bda3c9630e8616a/users/1801fc6e799747338a2cd99c300c8405/roles/9baff2b18f2649f7b757ff2e1f63c853 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:50:49.000173 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d627d531-c7df-469f-93e7-4b945ec863de admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:50:49.010155 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d627d531-c7df-469f-93e7-4b945ec863de admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:50:49.017944 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:50:49.065499 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a883954-a988-4ccc-96c9-2d41c42eb8a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:50:49.066173 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a883954-a988-4ccc-96c9-2d41c42eb8a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:49.066468 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a883954-a988-4ccc-96c9-2d41c42eb8a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:49.066748 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a883954-a988-4ccc-96c9-2d41c42eb8a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:49.067482 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a883954-a988-4ccc-96c9-2d41c42eb8a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:50:49.069179 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a883954-a988-4ccc-96c9-2d41c42eb8a0 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:50:49.079136 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1296/2563] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:50:48 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:50:49.083585 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f222f9d-192b-4ae1-8e87-1bde5e53cd1d admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:50:49.094802 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f222f9d-192b-4ae1-8e87-1bde5e53cd1d admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:50:49.100616 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fe0809-379d-4516-ac48-26eb208518fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:50:49.101507 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fe0809-379d-4516-ac48-26eb208518fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:49.101789 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fe0809-379d-4516-ac48-26eb208518fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:49.102132 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fe0809-379d-4516-ac48-26eb208518fc admin admin] PATH_INFO: `/v3/projects/021a562997dc4bec8bda3c9630e8616a/users/1801fc6e799747338a2cd99c300c8405/roles/9baff2b18f2649f7b757ff2e1f63c853` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:49.104093 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fe0809-379d-4516-ac48-26eb208518fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=021a562997dc4bec8bda3c9630e8616a, user_id=1801fc6e799747338a2cd99c300c8405, role_id=9baff2b18f2649f7b757ff2e1f63c853)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:50:49.106833 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fe0809-379d-4516-ac48-26eb208518fc admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:50:49.117536 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1268/2564] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:50:49 2022] PUT /identity/v3/projects/021a562997dc4bec8bda3c9630e8616a/users/1801fc6e799747338a2cd99c300c8405/roles/9baff2b18f2649f7b757ff2e1f63c853 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:50:49.133065 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d7ead-e923-4bca-b819-d306e3fcec63 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:49.133351 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d7ead-e923-4bca-b819-d306e3fcec63 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:49.133635 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d7ead-e923-4bca-b819-d306e3fcec63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:49.161482 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-924d7ead-e923-4bca-b819-d306e3fcec63 None None] MFA Rules not processed for user `1801fc6e799747338a2cd99c300c8405`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:50:49.163274 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-924d7ead-e923-4bca-b819-d306e3fcec63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:50:49.236342 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1297/2565] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:50:49 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:50:49.249221 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fe0809-379d-4516-ac48-26eb208518fc admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:50:49.310738 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fe0809-379d-4516-ac48-26eb208518fc admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:50:49.314482 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7bf316-3919-472f-bfc4-79ba8fd06c6a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d81f597cb44c56a966537089bcce4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:50:49.315150 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7bf316-3919-472f-bfc4-79ba8fd06c6a service cinder] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:49.315435 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7bf316-3919-472f-bfc4-79ba8fd06c6a service cinder] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:49.315710 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7bf316-3919-472f-bfc4-79ba8fd06c6a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:49.324808 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7bf316-3919-472f-bfc4-79ba8fd06c6a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:50:49.326217 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7bf316-3919-472f-bfc4-79ba8fd06c6a service cinder] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:50:49.336165 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1269/2566] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:50:49 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:50:50.375287 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924d7ead-e923-4bca-b819-d306e3fcec63 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:50:50.387652 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924d7ead-e923-4bca-b819-d306e3fcec63 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:50:50.394925 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17220c0-fb9a-43fd-8aa4-efa357a22e39 tempest-VolumeQuotaClassesTest-54780995 tempest-VolumeQuotaClassesTest-54780995-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1801fc6e799747338a2cd99c300c8405', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '021a562997dc4bec8bda3c9630e8616a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:50:50.395959 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17220c0-fb9a-43fd-8aa4-efa357a22e39 tempest-VolumeQuotaClassesTest-54780995 tempest-VolumeQuotaClassesTest-54780995-project-admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:50.397092 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17220c0-fb9a-43fd-8aa4-efa357a22e39 tempest-VolumeQuotaClassesTest-54780995 tempest-VolumeQuotaClassesTest-54780995-project-admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:50.397536 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17220c0-fb9a-43fd-8aa4-efa357a22e39 tempest-VolumeQuotaClassesTest-54780995 tempest-VolumeQuotaClassesTest-54780995-project-admin] PATH_INFO: `/v3/domains` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:50.398786 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17220c0-fb9a-43fd-8aa4-efa357a22e39 tempest-VolumeQuotaClassesTest-54780995 tempest-VolumeQuotaClassesTest-54780995-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:50:50.399352 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17220c0-fb9a-43fd-8aa4-efa357a22e39 tempest-VolumeQuotaClassesTest-54780995 tempest-VolumeQuotaClassesTest-54780995-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:50:50.401553 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17220c0-fb9a-43fd-8aa4-efa357a22e39 tempest-VolumeQuotaClassesTest-54780995 tempest-VolumeQuotaClassesTest-54780995-project-admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:50:50.413934 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1298/2567] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jun 7 14:50:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:50:50.418108 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7bf316-3919-472f-bfc4-79ba8fd06c6a service cinder] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:50:50.427219 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7bf316-3919-472f-bfc4-79ba8fd06c6a service cinder] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:50:50.431976 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad1a3bb-c34f-41ca-a191-388caa4f32d2 tempest-VolumeQuotaClassesTest-54780995 tempest-VolumeQuotaClassesTest-54780995-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1801fc6e799747338a2cd99c300c8405', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '021a562997dc4bec8bda3c9630e8616a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:50:50.432648 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad1a3bb-c34f-41ca-a191-388caa4f32d2 tempest-VolumeQuotaClassesTest-54780995 tempest-VolumeQuotaClassesTest-54780995-project-admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:50.432938 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad1a3bb-c34f-41ca-a191-388caa4f32d2 tempest-VolumeQuotaClassesTest-54780995 tempest-VolumeQuotaClassesTest-54780995-project-admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:50.433216 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad1a3bb-c34f-41ca-a191-388caa4f32d2 tempest-VolumeQuotaClassesTest-54780995 tempest-VolumeQuotaClassesTest-54780995-project-admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:50.433914 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad1a3bb-c34f-41ca-a191-388caa4f32d2 tempest-VolumeQuotaClassesTest-54780995 tempest-VolumeQuotaClassesTest-54780995-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:50:50.435778 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad1a3bb-c34f-41ca-a191-388caa4f32d2 tempest-VolumeQuotaClassesTest-54780995 tempest-VolumeQuotaClassesTest-54780995-project-admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:50:50.557219 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1270/2568] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:50:50 2022] POST /identity/v3/projects => generated 352 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:50:50.561438 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17220c0-fb9a-43fd-8aa4-efa357a22e39 tempest-VolumeQuotaClassesTest-54780995 tempest-VolumeQuotaClassesTest-54780995-project-admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:50:50.572435 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17220c0-fb9a-43fd-8aa4-efa357a22e39 tempest-VolumeQuotaClassesTest-54780995 tempest-VolumeQuotaClassesTest-54780995-project-admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:50:50.629002 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5125b5ea-7b20-4d9c-ac6e-7c392c33cd45 tempest-VolumeQuotaClassesTest-54780995 tempest-VolumeQuotaClassesTest-54780995-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1801fc6e799747338a2cd99c300c8405', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '021a562997dc4bec8bda3c9630e8616a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:50:50.629591 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5125b5ea-7b20-4d9c-ac6e-7c392c33cd45 tempest-VolumeQuotaClassesTest-54780995 tempest-VolumeQuotaClassesTest-54780995-project-admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:50.629885 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5125b5ea-7b20-4d9c-ac6e-7c392c33cd45 tempest-VolumeQuotaClassesTest-54780995 tempest-VolumeQuotaClassesTest-54780995-project-admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:50.630167 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5125b5ea-7b20-4d9c-ac6e-7c392c33cd45 tempest-VolumeQuotaClassesTest-54780995 tempest-VolumeQuotaClassesTest-54780995-project-admin] PATH_INFO: `/v3/domains` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:50.631490 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5125b5ea-7b20-4d9c-ac6e-7c392c33cd45 tempest-VolumeQuotaClassesTest-54780995 tempest-VolumeQuotaClassesTest-54780995-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:50:50.631804 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5125b5ea-7b20-4d9c-ac6e-7c392c33cd45 tempest-VolumeQuotaClassesTest-54780995 tempest-VolumeQuotaClassesTest-54780995-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:50:50.633451 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5125b5ea-7b20-4d9c-ac6e-7c392c33cd45 tempest-VolumeQuotaClassesTest-54780995 tempest-VolumeQuotaClassesTest-54780995-project-admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:50:50.645138 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1299/2569] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jun 7 14:50:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:50:50.683420 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad1a3bb-c34f-41ca-a191-388caa4f32d2 tempest-VolumeQuotaClassesTest-54780995 tempest-VolumeQuotaClassesTest-54780995-project-admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:50:50.695763 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad1a3bb-c34f-41ca-a191-388caa4f32d2 tempest-VolumeQuotaClassesTest-54780995 tempest-VolumeQuotaClassesTest-54780995-project-admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:50:50.700831 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a45ef3-f4a2-47ab-8378-00e24bfba672 tempest-VolumeQuotaClassesTest-54780995 tempest-VolumeQuotaClassesTest-54780995-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1801fc6e799747338a2cd99c300c8405', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '021a562997dc4bec8bda3c9630e8616a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:50:50.701823 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a45ef3-f4a2-47ab-8378-00e24bfba672 tempest-VolumeQuotaClassesTest-54780995 tempest-VolumeQuotaClassesTest-54780995-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:50.702268 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a45ef3-f4a2-47ab-8378-00e24bfba672 tempest-VolumeQuotaClassesTest-54780995 tempest-VolumeQuotaClassesTest-54780995-project-admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:50.702545 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a45ef3-f4a2-47ab-8378-00e24bfba672 tempest-VolumeQuotaClassesTest-54780995 tempest-VolumeQuotaClassesTest-54780995-project-admin] PATH_INFO: `/v3/projects/93b81421cbf442ecb515cb083ec1cb89` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:50.703945 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a45ef3-f4a2-47ab-8378-00e24bfba672 tempest-VolumeQuotaClassesTest-54780995 tempest-VolumeQuotaClassesTest-54780995-project-admin] RBAC: Authorizing `identity:delete_project(project_id=93b81421cbf442ecb515cb083ec1cb89)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:50:50.705725 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a45ef3-f4a2-47ab-8378-00e24bfba672 tempest-VolumeQuotaClassesTest-54780995 tempest-VolumeQuotaClassesTest-54780995-project-admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:50:50.842780 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-93a45ef3-f4a2-47ab-8378-00e24bfba672 tempest-VolumeQuotaClassesTest-54780995 tempest-VolumeQuotaClassesTest-54780995-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93b81421cbf442ecb515cb083ec1cb89', 'request_id': 'req-93a45ef3-f4a2-47ab-8378-00e24bfba672'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:50:50.847053 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=88,keystone:INSERT=7,keystone:DELETE=5 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:50:50.932257 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-93a45ef3-f4a2-47ab-8378-00e24bfba672 tempest-VolumeQuotaClassesTest-54780995 tempest-VolumeQuotaClassesTest-54780995-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93b81421cbf442ecb515cb083ec1cb89', 'request_id': 'req-93a45ef3-f4a2-47ab-8378-00e24bfba672'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:50:50.938371 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-93a45ef3-f4a2-47ab-8378-00e24bfba672 tempest-VolumeQuotaClassesTest-54780995 tempest-VolumeQuotaClassesTest-54780995-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93b81421cbf442ecb515cb083ec1cb89', 'request_id': 'req-93a45ef3-f4a2-47ab-8378-00e24bfba672'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:50:50.953446 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-93a45ef3-f4a2-47ab-8378-00e24bfba672 tempest-VolumeQuotaClassesTest-54780995 tempest-VolumeQuotaClassesTest-54780995-project-admin] The token cache is being invalidate because project 93b81421cbf442ecb515cb083ec1cb89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:50:50.953744 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-93a45ef3-f4a2-47ab-8378-00e24bfba672 tempest-VolumeQuotaClassesTest-54780995 tempest-VolumeQuotaClassesTest-54780995-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:50:50.954425 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-93a45ef3-f4a2-47ab-8378-00e24bfba672 tempest-VolumeQuotaClassesTest-54780995 tempest-VolumeQuotaClassesTest-54780995-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:50:50.955995 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1271/2570] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:50:50 2022] DELETE /identity/v3/projects/93b81421cbf442ecb515cb083ec1cb89 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:50:51.610484 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5125b5ea-7b20-4d9c-ac6e-7c392c33cd45 tempest-VolumeQuotaClassesTest-54780995 tempest-VolumeQuotaClassesTest-54780995-project-admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:50:51.612242 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-5125b5ea-7b20-4d9c-ac6e-7c392c33cd45 tempest-VolumeQuotaClassesTest-54780995 tempest-VolumeQuotaClassesTest-54780995-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:50:51.627349 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:50:51.679378 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5125b5ea-7b20-4d9c-ac6e-7c392c33cd45 tempest-VolumeQuotaClassesTest-54780995 tempest-VolumeQuotaClassesTest-54780995-project-admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:50:51.736443 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf1d0ae-e8aa-4157-80ad-9197cccd77eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:50:51.737256 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf1d0ae-e8aa-4157-80ad-9197cccd77eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:51.737547 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf1d0ae-e8aa-4157-80ad-9197cccd77eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:51.737831 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf1d0ae-e8aa-4157-80ad-9197cccd77eb admin admin] PATH_INFO: `/v3/users/85d84f05a1b44adfa4f0afb3f22c8fe4` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:51.739099 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf1d0ae-e8aa-4157-80ad-9197cccd77eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=85d84f05a1b44adfa4f0afb3f22c8fe4)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:50:51.741089 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf1d0ae-e8aa-4157-80ad-9197cccd77eb admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:50:51.903919 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-caf1d0ae-e8aa-4157-80ad-9197cccd77eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85d84f05a1b44adfa4f0afb3f22c8fe4'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:50:51.904907 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-caf1d0ae-e8aa-4157-80ad-9197cccd77eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85d84f05a1b44adfa4f0afb3f22c8fe4'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:50:51.905834 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-caf1d0ae-e8aa-4157-80ad-9197cccd77eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85d84f05a1b44adfa4f0afb3f22c8fe4'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:50:51.913419 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-caf1d0ae-e8aa-4157-80ad-9197cccd77eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85d84f05a1b44adfa4f0afb3f22c8fe4'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:50:51.925338 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-caf1d0ae-e8aa-4157-80ad-9197cccd77eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85d84f05a1b44adfa4f0afb3f22c8fe4'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:50:52.041146 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1300/2571] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:50:51 2022] DELETE /identity/v3/users/85d84f05a1b44adfa4f0afb3f22c8fe4 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:50:52.045748 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32420bc-dd5b-49c1-869d-1fe9dff395da None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:52.046234 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32420bc-dd5b-49c1-869d-1fe9dff395da None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:52.046333 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32420bc-dd5b-49c1-869d-1fe9dff395da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:52.063899 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-f32420bc-dd5b-49c1-869d-1fe9dff395da None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:50:52.065205 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-f32420bc-dd5b-49c1-869d-1fe9dff395da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:50:52.124385 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1272/2572] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:50:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:50:52.134801 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf1d0ae-e8aa-4157-80ad-9197cccd77eb admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:50:52.136274 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-caf1d0ae-e8aa-4157-80ad-9197cccd77eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:50:52.181290 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:50:52.195787 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf1d0ae-e8aa-4157-80ad-9197cccd77eb admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:50:52.199495 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dbe66c-b16b-4f3c-8e82-ceed6a26f698 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:50:52.200140 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dbe66c-b16b-4f3c-8e82-ceed6a26f698 service neutron] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:52.200461 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dbe66c-b16b-4f3c-8e82-ceed6a26f698 service neutron] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:52.200746 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dbe66c-b16b-4f3c-8e82-ceed6a26f698 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:52.209924 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dbe66c-b16b-4f3c-8e82-ceed6a26f698 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:50:52.211408 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dbe66c-b16b-4f3c-8e82-ceed6a26f698 service neutron] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:50:52.220841 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1301/2573] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:50:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:50:52.756900 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32420bc-dd5b-49c1-869d-1fe9dff395da None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:50:52.758323 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-f32420bc-dd5b-49c1-869d-1fe9dff395da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:50:52.771280 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32420bc-dd5b-49c1-869d-1fe9dff395da None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:50:52.777823 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eab950-0fa9-4530-92ae-f803af4c8003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:50:52.782074 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eab950-0fa9-4530-92ae-f803af4c8003 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:52.782074 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eab950-0fa9-4530-92ae-f803af4c8003 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:52.782074 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eab950-0fa9-4530-92ae-f803af4c8003 admin admin] PATH_INFO: `/v3/projects/126a4e5f998d4e588ebb72ad0187ca09` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:52.782074 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48eab950-0fa9-4530-92ae-f803af4c8003 admin admin] RBAC: Authorizing `identity:delete_project(project_id=126a4e5f998d4e588ebb72ad0187ca09)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:50:52.782813 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48eab950-0fa9-4530-92ae-f803af4c8003 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:50:52.892835 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-48eab950-0fa9-4530-92ae-f803af4c8003 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '126a4e5f998d4e588ebb72ad0187ca09', 'request_id': 'req-48eab950-0fa9-4530-92ae-f803af4c8003'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:50:52.983372 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-48eab950-0fa9-4530-92ae-f803af4c8003 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '126a4e5f998d4e588ebb72ad0187ca09', 'request_id': 'req-48eab950-0fa9-4530-92ae-f803af4c8003'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:50:52.989683 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-48eab950-0fa9-4530-92ae-f803af4c8003 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '126a4e5f998d4e588ebb72ad0187ca09', 'request_id': 'req-48eab950-0fa9-4530-92ae-f803af4c8003'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:50:53.001132 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-48eab950-0fa9-4530-92ae-f803af4c8003 admin admin] The token cache is being invalidate because project 126a4e5f998d4e588ebb72ad0187ca09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:50:53.001420 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-48eab950-0fa9-4530-92ae-f803af4c8003 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:50:53.002045 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-48eab950-0fa9-4530-92ae-f803af4c8003 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:50:53.003601 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1273/2574] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:50:52 2022] DELETE /identity/v3/projects/126a4e5f998d4e588ebb72ad0187ca09 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:50:53.007923 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dbe66c-b16b-4f3c-8e82-ceed6a26f698 service neutron] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:50:53.017710 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dbe66c-b16b-4f3c-8e82-ceed6a26f698 service neutron] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:50:53.074763 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1260e0-06ee-48e6-b15e-11f4c9cab335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:50:53.075641 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1260e0-06ee-48e6-b15e-11f4c9cab335 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:53.075929 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1260e0-06ee-48e6-b15e-11f4c9cab335 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:53.076212 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1260e0-06ee-48e6-b15e-11f4c9cab335 admin admin] PATH_INFO: `/v3/users/1801fc6e799747338a2cd99c300c8405` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:53.077440 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1260e0-06ee-48e6-b15e-11f4c9cab335 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1801fc6e799747338a2cd99c300c8405)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:50:53.079365 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1260e0-06ee-48e6-b15e-11f4c9cab335 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:50:53.245162 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-9a1260e0-06ee-48e6-b15e-11f4c9cab335 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1801fc6e799747338a2cd99c300c8405'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:50:53.245830 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-9a1260e0-06ee-48e6-b15e-11f4c9cab335 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1801fc6e799747338a2cd99c300c8405'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:50:53.248749 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-9a1260e0-06ee-48e6-b15e-11f4c9cab335 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1801fc6e799747338a2cd99c300c8405'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:50:53.253420 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-9a1260e0-06ee-48e6-b15e-11f4c9cab335 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1801fc6e799747338a2cd99c300c8405'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:50:53.265598 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-9a1260e0-06ee-48e6-b15e-11f4c9cab335 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1801fc6e799747338a2cd99c300c8405'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:50:53.326665 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1302/2575] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:50:53 2022] DELETE /identity/v3/users/1801fc6e799747338a2cd99c300c8405 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:50:53.767706 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eab950-0fa9-4530-92ae-f803af4c8003 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:50:53.769515 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-48eab950-0fa9-4530-92ae-f803af4c8003 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:50:53.783731 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eab950-0fa9-4530-92ae-f803af4c8003 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:50:53.842712 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b1daab-7067-49ab-8dfc-52a70326444c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:50:53.843584 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b1daab-7067-49ab-8dfc-52a70326444c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:53.843872 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b1daab-7067-49ab-8dfc-52a70326444c admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:53.844150 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b1daab-7067-49ab-8dfc-52a70326444c admin admin] PATH_INFO: `/v3/projects/021a562997dc4bec8bda3c9630e8616a` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:53.845365 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b1daab-7067-49ab-8dfc-52a70326444c admin admin] RBAC: Authorizing `identity:delete_project(project_id=021a562997dc4bec8bda3c9630e8616a)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:50:53.847300 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b1daab-7067-49ab-8dfc-52a70326444c admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:50:53.855785 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:50:53.951502 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-43b1daab-7067-49ab-8dfc-52a70326444c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '021a562997dc4bec8bda3c9630e8616a', 'request_id': 'req-43b1daab-7067-49ab-8dfc-52a70326444c'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:50:54.051970 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-43b1daab-7067-49ab-8dfc-52a70326444c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '021a562997dc4bec8bda3c9630e8616a', 'request_id': 'req-43b1daab-7067-49ab-8dfc-52a70326444c'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:50:54.058397 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-43b1daab-7067-49ab-8dfc-52a70326444c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '021a562997dc4bec8bda3c9630e8616a', 'request_id': 'req-43b1daab-7067-49ab-8dfc-52a70326444c'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:50:54.070985 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-43b1daab-7067-49ab-8dfc-52a70326444c admin admin] The token cache is being invalidate because project 021a562997dc4bec8bda3c9630e8616a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:50:54.071271 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-43b1daab-7067-49ab-8dfc-52a70326444c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:50:54.071916 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-43b1daab-7067-49ab-8dfc-52a70326444c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:50:54.073619 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1274/2576] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:50:53 2022] DELETE /identity/v3/projects/021a562997dc4bec8bda3c9630e8616a => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:50:54.078533 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196f16ef-f7ac-40b3-b376-eb277747602f None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:54.079012 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196f16ef-f7ac-40b3-b376-eb277747602f None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:54.079200 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196f16ef-f7ac-40b3-b376-eb277747602f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:54.097405 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-196f16ef-f7ac-40b3-b376-eb277747602f None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:50:54.099592 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-196f16ef-f7ac-40b3-b376-eb277747602f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:50:54.159332 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1303/2577] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:50:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:50:54.167718 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ca7b6-1e3b-46fc-a97c-fbdf7ea4fc19 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:54.168011 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ca7b6-1e3b-46fc-a97c-fbdf7ea4fc19 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:54.168311 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ca7b6-1e3b-46fc-a97c-fbdf7ea4fc19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:54.187250 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-442ca7b6-1e3b-46fc-a97c-fbdf7ea4fc19 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:50:54.189136 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-442ca7b6-1e3b-46fc-a97c-fbdf7ea4fc19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:50:54.196491 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1275/2578] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:50:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:50:54.200776 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196f16ef-f7ac-40b3-b376-eb277747602f None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:50:54.205637 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:50:54.211398 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196f16ef-f7ac-40b3-b376-eb277747602f None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:50:54.216096 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b1aecf-f5c2-4ad5-9ee1-a2e5ad873b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:50:54.216790 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b1aecf-f5c2-4ad5-9ee1-a2e5ad873b11 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:54.217075 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b1aecf-f5c2-4ad5-9ee1-a2e5ad873b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:54.217347 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b1aecf-f5c2-4ad5-9ee1-a2e5ad873b11 admin admin] PATH_INFO: `/v3/domains` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:54.218071 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b1aecf-f5c2-4ad5-9ee1-a2e5ad873b11 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:50:54.218376 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b1aecf-f5c2-4ad5-9ee1-a2e5ad873b11 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:50:54.219977 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b1aecf-f5c2-4ad5-9ee1-a2e5ad873b11 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:50:54.232968 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1304/2579] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jun 7 14:50:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:50:54.237621 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442ca7b6-1e3b-46fc-a97c-fbdf7ea4fc19 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:50:54.247980 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442ca7b6-1e3b-46fc-a97c-fbdf7ea4fc19 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:50:54.252564 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d95d11-e53e-4023-b0ba-02df0ab3bc3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:50:54.253234 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d95d11-e53e-4023-b0ba-02df0ab3bc3d admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:54.253522 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d95d11-e53e-4023-b0ba-02df0ab3bc3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:54.253800 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d95d11-e53e-4023-b0ba-02df0ab3bc3d admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:54.254734 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d95d11-e53e-4023-b0ba-02df0ab3bc3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:50:54.256566 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d95d11-e53e-4023-b0ba-02df0ab3bc3d admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:50:54.344047 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1276/2580] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:50:54 2022] POST /identity/v3/projects => generated 375 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:50:54.348342 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b1aecf-f5c2-4ad5-9ee1-a2e5ad873b11 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:50:54.358953 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b1aecf-f5c2-4ad5-9ee1-a2e5ad873b11 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:50:54.417791 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a54b80-9677-466e-9d64-0e6dc994c70a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:50:54.418564 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a54b80-9677-466e-9d64-0e6dc994c70a admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:54.418885 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a54b80-9677-466e-9d64-0e6dc994c70a admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:54.419361 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a54b80-9677-466e-9d64-0e6dc994c70a admin admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:54.420900 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a54b80-9677-466e-9d64-0e6dc994c70a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:50:54.422599 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a54b80-9677-466e-9d64-0e6dc994c70a admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:50:54.509428 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1305/2581] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:50:54 2022] POST /identity/v3/users => generated 288 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:50:54.513562 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d95d11-e53e-4023-b0ba-02df0ab3bc3d admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:50:54.524668 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d95d11-e53e-4023-b0ba-02df0ab3bc3d admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:50:54.529872 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c971d9-bba6-411f-828a-fb1119508baf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:50:54.530671 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c971d9-bba6-411f-828a-fb1119508baf admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:54.530957 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c971d9-bba6-411f-828a-fb1119508baf admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:54.531234 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c971d9-bba6-411f-828a-fb1119508baf admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:54.531958 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c971d9-bba6-411f-828a-fb1119508baf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:50:54.533658 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c971d9-bba6-411f-828a-fb1119508baf admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:50:54.547275 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1277/2582] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:50:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:50:54.551939 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a54b80-9677-466e-9d64-0e6dc994c70a admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:50:54.562629 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a54b80-9677-466e-9d64-0e6dc994c70a admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:50:54.567624 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861c7d43-1242-4c09-9af8-2111469a8f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:50:54.568425 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861c7d43-1242-4c09-9af8-2111469a8f81 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:54.568726 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861c7d43-1242-4c09-9af8-2111469a8f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:54.569010 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861c7d43-1242-4c09-9af8-2111469a8f81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:54.569751 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861c7d43-1242-4c09-9af8-2111469a8f81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:50:54.571360 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861c7d43-1242-4c09-9af8-2111469a8f81 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:50:54.584868 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1306/2583] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:50:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:50:54.587878 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c971d9-bba6-411f-828a-fb1119508baf admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:50:54.598944 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c971d9-bba6-411f-828a-fb1119508baf admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:50:54.604953 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8675adad-eb37-4555-859d-1c8000cee942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:50:54.606298 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8675adad-eb37-4555-859d-1c8000cee942 admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:54.606587 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8675adad-eb37-4555-859d-1c8000cee942 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:54.606867 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8675adad-eb37-4555-859d-1c8000cee942 admin admin] PATH_INFO: `/v3/projects/df188116239040b99db6f9400fb1ccfa/users/20a8205d4f7443d3b130f0aaee0da445/roles/3771341b3be641139e752aab690397a4` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:54.625574 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8675adad-eb37-4555-859d-1c8000cee942 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df188116239040b99db6f9400fb1ccfa, user_id=20a8205d4f7443d3b130f0aaee0da445, role_id=3771341b3be641139e752aab690397a4)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:50:54.628328 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8675adad-eb37-4555-859d-1c8000cee942 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:50:54.708779 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1278/2584] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:50:54 2022] PUT /identity/v3/projects/df188116239040b99db6f9400fb1ccfa/users/20a8205d4f7443d3b130f0aaee0da445/roles/3771341b3be641139e752aab690397a4 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:50:54.721197 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b392f335-e7d2-4f5b-bddf-ccc6b0a77ab5 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:54.721510 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b392f335-e7d2-4f5b-bddf-ccc6b0a77ab5 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:54.721778 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b392f335-e7d2-4f5b-bddf-ccc6b0a77ab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:54.747699 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-b392f335-e7d2-4f5b-bddf-ccc6b0a77ab5 None None] MFA Rules not processed for user `20a8205d4f7443d3b130f0aaee0da445`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:50:54.748951 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-b392f335-e7d2-4f5b-bddf-ccc6b0a77ab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:50:54.812125 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1307/2585] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:50:54 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:50:54.816681 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8675adad-eb37-4555-859d-1c8000cee942 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:50:54.879574 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8675adad-eb37-4555-859d-1c8000cee942 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:50:54.884792 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9e6200-abb7-44d4-93af-8d2aead1ed0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:50:54.889244 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9e6200-abb7-44d4-93af-8d2aead1ed0f admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:54.889244 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9e6200-abb7-44d4-93af-8d2aead1ed0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:54.889244 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9e6200-abb7-44d4-93af-8d2aead1ed0f admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:54.889244 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9e6200-abb7-44d4-93af-8d2aead1ed0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:50:54.889244 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9e6200-abb7-44d4-93af-8d2aead1ed0f admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:50:54.979854 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1279/2586] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:50:54 2022] POST /identity/v3/projects => generated 373 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:50:54.983905 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b392f335-e7d2-4f5b-bddf-ccc6b0a77ab5 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:50:54.995185 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b392f335-e7d2-4f5b-bddf-ccc6b0a77ab5 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:50:55.044721 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:50:55.050955 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d1617c-352b-4feb-851e-e8e7dd692059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:50:55.051646 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d1617c-352b-4feb-851e-e8e7dd692059 admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:55.051934 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d1617c-352b-4feb-851e-e8e7dd692059 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:55.052215 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d1617c-352b-4feb-851e-e8e7dd692059 admin admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:55.052965 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d1617c-352b-4feb-851e-e8e7dd692059 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:50:55.055513 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d1617c-352b-4feb-851e-e8e7dd692059 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:50:55.143615 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1308/2587] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:50:54 2022] POST /identity/v3/users => generated 293 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:50:55.147975 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9e6200-abb7-44d4-93af-8d2aead1ed0f admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:50:55.158296 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9e6200-abb7-44d4-93af-8d2aead1ed0f admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:50:55.162956 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c560f0-54e1-4b99-88b3-94df98467552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:50:55.163610 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c560f0-54e1-4b99-88b3-94df98467552 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:55.163898 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c560f0-54e1-4b99-88b3-94df98467552 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:55.164180 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c560f0-54e1-4b99-88b3-94df98467552 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:55.164931 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c560f0-54e1-4b99-88b3-94df98467552 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:50:55.170059 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c560f0-54e1-4b99-88b3-94df98467552 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:50:55.177030 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1280/2588] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:50:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:50:55.182175 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d1617c-352b-4feb-851e-e8e7dd692059 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:50:55.192780 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d1617c-352b-4feb-851e-e8e7dd692059 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:50:55.198079 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06d8dc1-6d2b-4164-837b-9fa91cf775c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:50:55.199012 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06d8dc1-6d2b-4164-837b-9fa91cf775c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:55.199299 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06d8dc1-6d2b-4164-837b-9fa91cf775c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:55.199578 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06d8dc1-6d2b-4164-837b-9fa91cf775c1 admin admin] PATH_INFO: `/v3/projects/dbe6b96032ed44f1a17afe0041739ed6/users/f03224843cdb4d79b93280c9ac3dbc0f/roles/9baff2b18f2649f7b757ff2e1f63c853` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:55.217066 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06d8dc1-6d2b-4164-837b-9fa91cf775c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbe6b96032ed44f1a17afe0041739ed6, user_id=f03224843cdb4d79b93280c9ac3dbc0f, role_id=9baff2b18f2649f7b757ff2e1f63c853)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:50:55.221110 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06d8dc1-6d2b-4164-837b-9fa91cf775c1 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:50:55.338653 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1309/2589] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:50:55 2022] PUT /identity/v3/projects/dbe6b96032ed44f1a17afe0041739ed6/users/f03224843cdb4d79b93280c9ac3dbc0f/roles/9baff2b18f2649f7b757ff2e1f63c853 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:50:55.342744 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c560f0-54e1-4b99-88b3-94df98467552 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:50:55.353482 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c560f0-54e1-4b99-88b3-94df98467552 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:50:55.357073 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:50:55.410879 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eadbe70-8887-4eda-a659-08f66b0518e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:50:55.411459 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eadbe70-8887-4eda-a659-08f66b0518e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:55.411739 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eadbe70-8887-4eda-a659-08f66b0518e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:55.412020 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eadbe70-8887-4eda-a659-08f66b0518e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:55.412766 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eadbe70-8887-4eda-a659-08f66b0518e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:50:55.414351 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eadbe70-8887-4eda-a659-08f66b0518e1 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:50:55.423825 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1281/2590] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:50:55 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 14:50:55.428082 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06d8dc1-6d2b-4164-837b-9fa91cf775c1 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:50:55.438097 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06d8dc1-6d2b-4164-837b-9fa91cf775c1 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:50:55.442693 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8efc210-d427-4ec0-b735-5a174f764cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:50:55.443570 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8efc210-d427-4ec0-b735-5a174f764cea admin admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:55.444546 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8efc210-d427-4ec0-b735-5a174f764cea admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:55.444717 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8efc210-d427-4ec0-b735-5a174f764cea admin admin] PATH_INFO: `/v3/projects/dbe6b96032ed44f1a17afe0041739ed6/users/f03224843cdb4d79b93280c9ac3dbc0f/roles/9baff2b18f2649f7b757ff2e1f63c853` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:55.446663 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8efc210-d427-4ec0-b735-5a174f764cea admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbe6b96032ed44f1a17afe0041739ed6, user_id=f03224843cdb4d79b93280c9ac3dbc0f, role_id=9baff2b18f2649f7b757ff2e1f63c853)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:50:55.449288 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8efc210-d427-4ec0-b735-5a174f764cea admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:50:55.460034 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1310/2591] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:50:55 2022] PUT /identity/v3/projects/dbe6b96032ed44f1a17afe0041739ed6/users/f03224843cdb4d79b93280c9ac3dbc0f/roles/9baff2b18f2649f7b757ff2e1f63c853 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:50:55.470795 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac6dbaa-1864-4126-bf26-5b10e8122452 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:55.471087 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac6dbaa-1864-4126-bf26-5b10e8122452 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:55.471368 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac6dbaa-1864-4126-bf26-5b10e8122452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:55.500222 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-4ac6dbaa-1864-4126-bf26-5b10e8122452 None None] MFA Rules not processed for user `f03224843cdb4d79b93280c9ac3dbc0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:50:55.501875 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-4ac6dbaa-1864-4126-bf26-5b10e8122452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:50:55.572549 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1282/2592] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:50:55 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:50:55.583126 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8efc210-d427-4ec0-b735-5a174f764cea admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:50:55.585554 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-e8efc210-d427-4ec0-b735-5a174f764cea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:50:55.645155 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8efc210-d427-4ec0-b735-5a174f764cea admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:50:55.649422 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c3a1d8-65bc-439e-8644-4e7f9f577df8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d81f597cb44c56a966537089bcce4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:50:55.650151 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c3a1d8-65bc-439e-8644-4e7f9f577df8 service cinder] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:55.650434 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c3a1d8-65bc-439e-8644-4e7f9f577df8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:55.650716 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c3a1d8-65bc-439e-8644-4e7f9f577df8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:55.705534 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c3a1d8-65bc-439e-8644-4e7f9f577df8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:50:55.706982 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c3a1d8-65bc-439e-8644-4e7f9f577df8 service cinder] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:50:55.716545 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1311/2593] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:50:55 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:50:59.892281 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac6dbaa-1864-4126-bf26-5b10e8122452 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:50:59.954622 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac6dbaa-1864-4126-bf26-5b10e8122452 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:50:59.959287 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702b22a0-6755-4954-b429-6ac6f5948398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:50:59.960324 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702b22a0-6755-4954-b429-6ac6f5948398 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:50:59.960511 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702b22a0-6755-4954-b429-6ac6f5948398 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:50:59.960837 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702b22a0-6755-4954-b429-6ac6f5948398 admin admin] PATH_INFO: `/v3/users/20a8205d4f7443d3b130f0aaee0da445` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:50:59.961950 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702b22a0-6755-4954-b429-6ac6f5948398 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20a8205d4f7443d3b130f0aaee0da445)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:50:59.963869 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702b22a0-6755-4954-b429-6ac6f5948398 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:00.144421 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:51:00.158008 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-702b22a0-6755-4954-b429-6ac6f5948398 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20a8205d4f7443d3b130f0aaee0da445'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:00.158695 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-702b22a0-6755-4954-b429-6ac6f5948398 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20a8205d4f7443d3b130f0aaee0da445'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:00.159298 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-702b22a0-6755-4954-b429-6ac6f5948398 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20a8205d4f7443d3b130f0aaee0da445'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:00.166841 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-702b22a0-6755-4954-b429-6ac6f5948398 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20a8205d4f7443d3b130f0aaee0da445'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:00.179143 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-702b22a0-6755-4954-b429-6ac6f5948398 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20a8205d4f7443d3b130f0aaee0da445'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:00.264831 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1283/2594] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:50:59 2022] DELETE /identity/v3/users/20a8205d4f7443d3b130f0aaee0da445 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:51:00.269303 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61092b5-b7c4-4c55-98d6-ff43274916b6 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:00.270046 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61092b5-b7c4-4c55-98d6-ff43274916b6 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:00.270046 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61092b5-b7c4-4c55-98d6-ff43274916b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:00.288019 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-a61092b5-b7c4-4c55-98d6-ff43274916b6 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:51:00.289916 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-a61092b5-b7c4-4c55-98d6-ff43274916b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:51:00.349105 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1312/2595] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:51:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:51:00.359266 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702b22a0-6755-4954-b429-6ac6f5948398 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:00.360785 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-702b22a0-6755-4954-b429-6ac6f5948398 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:51:00.420662 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702b22a0-6755-4954-b429-6ac6f5948398 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:00.424434 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42533da5-ac51-40c9-9728-275c84c8611b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:00.425087 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42533da5-ac51-40c9-9728-275c84c8611b service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:00.425375 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42533da5-ac51-40c9-9728-275c84c8611b service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:00.425656 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42533da5-ac51-40c9-9728-275c84c8611b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:00.434830 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42533da5-ac51-40c9-9728-275c84c8611b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:00.440123 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42533da5-ac51-40c9-9728-275c84c8611b service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:00.446542 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1284/2596] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:51:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:51:00.922293 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61092b5-b7c4-4c55-98d6-ff43274916b6 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:00.924119 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-a61092b5-b7c4-4c55-98d6-ff43274916b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:51:00.934837 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:51:00.944026 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61092b5-b7c4-4c55-98d6-ff43274916b6 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:00.950648 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbb5569-406b-40c0-8b53-d9c964d0f720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:00.951611 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb5569-406b-40c0-8b53-d9c964d0f720 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:00.951801 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb5569-406b-40c0-8b53-d9c964d0f720 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:00.952081 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb5569-406b-40c0-8b53-d9c964d0f720 admin admin] PATH_INFO: `/v3/projects/df188116239040b99db6f9400fb1ccfa` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:00.953479 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbb5569-406b-40c0-8b53-d9c964d0f720 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df188116239040b99db6f9400fb1ccfa)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:00.955433 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbb5569-406b-40c0-8b53-d9c964d0f720 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:01.073043 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-3dbb5569-406b-40c0-8b53-d9c964d0f720 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df188116239040b99db6f9400fb1ccfa', 'request_id': 'req-3dbb5569-406b-40c0-8b53-d9c964d0f720'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:01.149216 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-3dbb5569-406b-40c0-8b53-d9c964d0f720 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df188116239040b99db6f9400fb1ccfa', 'request_id': 'req-3dbb5569-406b-40c0-8b53-d9c964d0f720'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:01.155585 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-3dbb5569-406b-40c0-8b53-d9c964d0f720 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df188116239040b99db6f9400fb1ccfa', 'request_id': 'req-3dbb5569-406b-40c0-8b53-d9c964d0f720'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:01.166896 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-3dbb5569-406b-40c0-8b53-d9c964d0f720 admin admin] The token cache is being invalidate because project df188116239040b99db6f9400fb1ccfa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:51:01.167185 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-3dbb5569-406b-40c0-8b53-d9c964d0f720 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:01.167826 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-3dbb5569-406b-40c0-8b53-d9c964d0f720 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:01.169422 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1313/2597] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:51:00 2022] DELETE /identity/v3/projects/df188116239040b99db6f9400fb1ccfa => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:51:01.173795 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42533da5-ac51-40c9-9728-275c84c8611b service neutron] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:01.185659 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42533da5-ac51-40c9-9728-275c84c8611b service neutron] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:01.244153 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaef210f-6d66-4c4c-8528-909763a2228e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:01.245166 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef210f-6d66-4c4c-8528-909763a2228e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:01.245534 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef210f-6d66-4c4c-8528-909763a2228e admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:01.245813 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef210f-6d66-4c4c-8528-909763a2228e admin admin] PATH_INFO: `/v3/users/f03224843cdb4d79b93280c9ac3dbc0f` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:01.247117 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaef210f-6d66-4c4c-8528-909763a2228e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f03224843cdb4d79b93280c9ac3dbc0f)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:01.249289 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaef210f-6d66-4c4c-8528-909763a2228e admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:01.376496 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbb5569-406b-40c0-8b53-d9c964d0f720 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:01.378339 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-3dbb5569-406b-40c0-8b53-d9c964d0f720 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:51:01.392547 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbb5569-406b-40c0-8b53-d9c964d0f720 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:01.397171 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae09321-1023-4725-ba5d-9ba287a75514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:01.398075 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae09321-1023-4725-ba5d-9ba287a75514 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:01.398358 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae09321-1023-4725-ba5d-9ba287a75514 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:01.398742 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae09321-1023-4725-ba5d-9ba287a75514 admin admin] PATH_INFO: `/v3/users/5f4058d42c0e467f81d709ac42f7ef94` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:01.399984 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae09321-1023-4725-ba5d-9ba287a75514 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f4058d42c0e467f81d709ac42f7ef94)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:01.402269 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae09321-1023-4725-ba5d-9ba287a75514 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:01.453849 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-eaef210f-6d66-4c4c-8528-909763a2228e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f03224843cdb4d79b93280c9ac3dbc0f'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:01.454516 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-eaef210f-6d66-4c4c-8528-909763a2228e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f03224843cdb4d79b93280c9ac3dbc0f'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:01.455147 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-eaef210f-6d66-4c4c-8528-909763a2228e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f03224843cdb4d79b93280c9ac3dbc0f'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:01.459851 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:51:01.462925 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-eaef210f-6d66-4c4c-8528-909763a2228e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f03224843cdb4d79b93280c9ac3dbc0f'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:01.474884 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-eaef210f-6d66-4c4c-8528-909763a2228e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f03224843cdb4d79b93280c9ac3dbc0f'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:01.537630 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1285/2598] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:51:01 2022] DELETE /identity/v3/users/f03224843cdb4d79b93280c9ac3dbc0f => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:51:01.576995 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-2ae09321-1023-4725-ba5d-9ba287a75514 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f4058d42c0e467f81d709ac42f7ef94'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:01.577717 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-2ae09321-1023-4725-ba5d-9ba287a75514 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f4058d42c0e467f81d709ac42f7ef94'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:01.578340 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-2ae09321-1023-4725-ba5d-9ba287a75514 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f4058d42c0e467f81d709ac42f7ef94'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:01.585562 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-2ae09321-1023-4725-ba5d-9ba287a75514 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f4058d42c0e467f81d709ac42f7ef94'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:01.597936 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-2ae09321-1023-4725-ba5d-9ba287a75514 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f4058d42c0e467f81d709ac42f7ef94'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:01.646613 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1314/2599] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:51:01 2022] DELETE /identity/v3/users/5f4058d42c0e467f81d709ac42f7ef94 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:51:01.653498 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246b6565-4d5c-4d85-958c-5f6ddbb14b54 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:01.653498 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246b6565-4d5c-4d85-958c-5f6ddbb14b54 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:01.653498 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246b6565-4d5c-4d85-958c-5f6ddbb14b54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:01.671439 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-246b6565-4d5c-4d85-958c-5f6ddbb14b54 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:51:01.672769 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-246b6565-4d5c-4d85-958c-5f6ddbb14b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:51:01.733255 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1286/2600] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:51:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:51:01.743966 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae09321-1023-4725-ba5d-9ba287a75514 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:01.745505 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-2ae09321-1023-4725-ba5d-9ba287a75514 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:51:01.814875 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae09321-1023-4725-ba5d-9ba287a75514 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:01.818810 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519a3e7c-6416-4210-aa04-181c1313da9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:01.819409 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519a3e7c-6416-4210-aa04-181c1313da9f service neutron] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:01.819679 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519a3e7c-6416-4210-aa04-181c1313da9f service neutron] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:01.820626 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519a3e7c-6416-4210-aa04-181c1313da9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:01.830038 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519a3e7c-6416-4210-aa04-181c1313da9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:01.831477 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519a3e7c-6416-4210-aa04-181c1313da9f service neutron] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:01.843045 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1315/2601] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:51:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:51:01.980395 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246b6565-4d5c-4d85-958c-5f6ddbb14b54 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:01.982001 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-246b6565-4d5c-4d85-958c-5f6ddbb14b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:51:01.994907 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246b6565-4d5c-4d85-958c-5f6ddbb14b54 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:02.004576 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b367b2ab-45b7-466c-9a1e-b5328e909f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:02.006024 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b367b2ab-45b7-466c-9a1e-b5328e909f9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:02.006522 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b367b2ab-45b7-466c-9a1e-b5328e909f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:02.007483 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b367b2ab-45b7-466c-9a1e-b5328e909f9e admin admin] PATH_INFO: `/v3/projects/dbe6b96032ed44f1a17afe0041739ed6` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:02.008731 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b367b2ab-45b7-466c-9a1e-b5328e909f9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbe6b96032ed44f1a17afe0041739ed6)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:02.010759 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b367b2ab-45b7-466c-9a1e-b5328e909f9e admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:02.159003 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-b367b2ab-45b7-466c-9a1e-b5328e909f9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbe6b96032ed44f1a17afe0041739ed6', 'request_id': 'req-b367b2ab-45b7-466c-9a1e-b5328e909f9e'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:02.239594 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-b367b2ab-45b7-466c-9a1e-b5328e909f9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbe6b96032ed44f1a17afe0041739ed6', 'request_id': 'req-b367b2ab-45b7-466c-9a1e-b5328e909f9e'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:02.246674 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-b367b2ab-45b7-466c-9a1e-b5328e909f9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbe6b96032ed44f1a17afe0041739ed6', 'request_id': 'req-b367b2ab-45b7-466c-9a1e-b5328e909f9e'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:02.263980 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-b367b2ab-45b7-466c-9a1e-b5328e909f9e admin admin] The token cache is being invalidate because project dbe6b96032ed44f1a17afe0041739ed6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:51:02.264609 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-b367b2ab-45b7-466c-9a1e-b5328e909f9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:02.265551 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-b367b2ab-45b7-466c-9a1e-b5328e909f9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:02.267774 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1287/2602] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:51:01 2022] DELETE /identity/v3/projects/dbe6b96032ed44f1a17afe0041739ed6 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:51:02.274764 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d196afe9-4e45-4bc3-818e-0423c50cebbd None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:02.275053 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d196afe9-4e45-4bc3-818e-0423c50cebbd None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:02.275329 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d196afe9-4e45-4bc3-818e-0423c50cebbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:02.282423 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:51:02.294827 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-d196afe9-4e45-4bc3-818e-0423c50cebbd None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:51:02.296173 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-d196afe9-4e45-4bc3-818e-0423c50cebbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:51:02.339865 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b367b2ab-45b7-466c-9a1e-b5328e909f9e admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:02.341542 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-b367b2ab-45b7-466c-9a1e-b5328e909f9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:51:02.375469 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1316/2603] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:51:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:51:02.382952 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63f4068-db54-4574-8a30-1c98ed515b14 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:02.383595 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63f4068-db54-4574-8a30-1c98ed515b14 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:02.383595 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63f4068-db54-4574-8a30-1c98ed515b14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:02.404681 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-c63f4068-db54-4574-8a30-1c98ed515b14 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:51:02.406404 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-c63f4068-db54-4574-8a30-1c98ed515b14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:51:02.411956 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:51:02.419391 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1317/2604] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:51:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:51:02.424107 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63f4068-db54-4574-8a30-1c98ed515b14 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:02.428433 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b367b2ab-45b7-466c-9a1e-b5328e909f9e admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:02.430332 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a3db8f-d2e3-457c-8aa2-86f7b8ba2e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:02.431209 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a3db8f-d2e3-457c-8aa2-86f7b8ba2e88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:02.431494 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a3db8f-d2e3-457c-8aa2-86f7b8ba2e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:02.431770 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a3db8f-d2e3-457c-8aa2-86f7b8ba2e88 admin admin] PATH_INFO: `/v3/projects/b803381e2ff74a1883ce59917d7e70ea` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:02.433013 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a3db8f-d2e3-457c-8aa2-86f7b8ba2e88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b803381e2ff74a1883ce59917d7e70ea)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:02.434961 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a3db8f-d2e3-457c-8aa2-86f7b8ba2e88 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:02.438050 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63f4068-db54-4574-8a30-1c98ed515b14 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:02.444696 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0127b57d-54fc-46e6-bf6e-50bea40d1569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:02.446255 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0127b57d-54fc-46e6-bf6e-50bea40d1569 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:02.446702 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0127b57d-54fc-46e6-bf6e-50bea40d1569 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:02.447139 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0127b57d-54fc-46e6-bf6e-50bea40d1569 admin admin] PATH_INFO: `/v3/domains` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:02.448262 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0127b57d-54fc-46e6-bf6e-50bea40d1569 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:51:02.448736 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0127b57d-54fc-46e6-bf6e-50bea40d1569 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:02.451393 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0127b57d-54fc-46e6-bf6e-50bea40d1569 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:02.464207 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1318/2605] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jun 7 14:51:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:51:02.470367 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0127b57d-54fc-46e6-bf6e-50bea40d1569 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:02.485334 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0127b57d-54fc-46e6-bf6e-50bea40d1569 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:02.490199 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254e2608-2b55-4e57-87b3-b5e4f4dee619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:02.490876 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254e2608-2b55-4e57-87b3-b5e4f4dee619 admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:02.491158 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254e2608-2b55-4e57-87b3-b5e4f4dee619 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:02.491439 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254e2608-2b55-4e57-87b3-b5e4f4dee619 admin admin] PATH_INFO: `/v3/projects` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:02.492684 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254e2608-2b55-4e57-87b3-b5e4f4dee619 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:02.494560 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254e2608-2b55-4e57-87b3-b5e4f4dee619 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:02.549475 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-47a3db8f-d2e3-457c-8aa2-86f7b8ba2e88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b803381e2ff74a1883ce59917d7e70ea', 'request_id': 'req-47a3db8f-d2e3-457c-8aa2-86f7b8ba2e88'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:02.618151 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1319/2606] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:51:02 2022] POST /identity/v3/projects => generated 371 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:51:02.622501 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254e2608-2b55-4e57-87b3-b5e4f4dee619 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:02.639982 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-47a3db8f-d2e3-457c-8aa2-86f7b8ba2e88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b803381e2ff74a1883ce59917d7e70ea', 'request_id': 'req-47a3db8f-d2e3-457c-8aa2-86f7b8ba2e88'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:02.647576 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-47a3db8f-d2e3-457c-8aa2-86f7b8ba2e88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b803381e2ff74a1883ce59917d7e70ea', 'request_id': 'req-47a3db8f-d2e3-457c-8aa2-86f7b8ba2e88'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:02.659396 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-47a3db8f-d2e3-457c-8aa2-86f7b8ba2e88 admin admin] The token cache is being invalidate because project b803381e2ff74a1883ce59917d7e70ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:51:02.659810 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-47a3db8f-d2e3-457c-8aa2-86f7b8ba2e88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:02.660532 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-47a3db8f-d2e3-457c-8aa2-86f7b8ba2e88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:02.667265 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1288/2607] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:51:02 2022] DELETE /identity/v3/projects/b803381e2ff74a1883ce59917d7e70ea => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:51:02.671625 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a3db8f-d2e3-457c-8aa2-86f7b8ba2e88 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:02.673085 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-47a3db8f-d2e3-457c-8aa2-86f7b8ba2e88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:51:02.694789 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254e2608-2b55-4e57-87b3-b5e4f4dee619 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:02.700247 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1c8565-9fec-4869-b29f-5758c0a67e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:02.700949 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1c8565-9fec-4869-b29f-5758c0a67e2e admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:02.701234 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1c8565-9fec-4869-b29f-5758c0a67e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:02.701513 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1c8565-9fec-4869-b29f-5758c0a67e2e admin admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:02.702204 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1c8565-9fec-4869-b29f-5758c0a67e2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:02.704248 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1c8565-9fec-4869-b29f-5758c0a67e2e admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:02.750582 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a3db8f-d2e3-457c-8aa2-86f7b8ba2e88 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:02.754201 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b149d3-b795-41e6-a46c-f51a73a905bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:02.755093 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b149d3-b795-41e6-a46c-f51a73a905bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:02.755379 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b149d3-b795-41e6-a46c-f51a73a905bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:02.755658 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b149d3-b795-41e6-a46c-f51a73a905bb admin admin] PATH_INFO: `/v3/users/ace643dc2641461ead9b135ba23087aa` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:02.757047 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b149d3-b795-41e6-a46c-f51a73a905bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=ace643dc2641461ead9b135ba23087aa)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:02.761129 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b149d3-b795-41e6-a46c-f51a73a905bb admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:02.795703 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1320/2608] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:51:02 2022] POST /identity/v3/users => generated 286 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:51:02.799898 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1c8565-9fec-4869-b29f-5758c0a67e2e admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:02.801595 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-cb1c8565-9fec-4869-b29f-5758c0a67e2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:51:02.813740 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1c8565-9fec-4869-b29f-5758c0a67e2e admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:02.818618 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215d4bdd-77f3-41c1-a7ee-a019eb37e4ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:02.819268 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215d4bdd-77f3-41c1-a7ee-a019eb37e4ea admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:02.819556 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215d4bdd-77f3-41c1-a7ee-a019eb37e4ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:02.820334 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215d4bdd-77f3-41c1-a7ee-a019eb37e4ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:02.821101 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215d4bdd-77f3-41c1-a7ee-a019eb37e4ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:02.822740 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215d4bdd-77f3-41c1-a7ee-a019eb37e4ea admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:02.833122 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1321/2609] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:51:02 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:51:02.838085 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215d4bdd-77f3-41c1-a7ee-a019eb37e4ea admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:02.847747 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215d4bdd-77f3-41c1-a7ee-a019eb37e4ea admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:02.852356 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8452fe74-6e48-42fc-9c89-f815dc923059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:02.853027 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8452fe74-6e48-42fc-9c89-f815dc923059 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:02.853310 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8452fe74-6e48-42fc-9c89-f815dc923059 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:02.853587 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8452fe74-6e48-42fc-9c89-f815dc923059 admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:02.854304 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8452fe74-6e48-42fc-9c89-f815dc923059 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:02.855882 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8452fe74-6e48-42fc-9c89-f815dc923059 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:02.866965 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1322/2610] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:51:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 14:51:02.879434 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8452fe74-6e48-42fc-9c89-f815dc923059 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:02.889704 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8452fe74-6e48-42fc-9c89-f815dc923059 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:02.894601 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6bfff8-d045-4d01-9816-a4cd649eabd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:02.895471 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6bfff8-d045-4d01-9816-a4cd649eabd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:02.895761 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6bfff8-d045-4d01-9816-a4cd649eabd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:02.896075 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6bfff8-d045-4d01-9816-a4cd649eabd4 admin admin] PATH_INFO: `/v3/projects/8c1eee16d77d46528ba44f3cad4c5095/users/e1c23b5f03f6445e9a10b93fb775c60c/roles/3771341b3be641139e752aab690397a4` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:02.911903 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6bfff8-d045-4d01-9816-a4cd649eabd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c1eee16d77d46528ba44f3cad4c5095, user_id=e1c23b5f03f6445e9a10b93fb775c60c, role_id=3771341b3be641139e752aab690397a4)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:02.914583 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6bfff8-d045-4d01-9816-a4cd649eabd4 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:02.979825 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a0b149d3-b795-41e6-a46c-f51a73a905bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ace643dc2641461ead9b135ba23087aa'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:02.980520 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a0b149d3-b795-41e6-a46c-f51a73a905bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ace643dc2641461ead9b135ba23087aa'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:02.981106 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a0b149d3-b795-41e6-a46c-f51a73a905bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ace643dc2641461ead9b135ba23087aa'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:02.988197 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a0b149d3-b795-41e6-a46c-f51a73a905bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ace643dc2641461ead9b135ba23087aa'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:02.999787 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1323/2611] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:51:02 2022] PUT /identity/v3/projects/8c1eee16d77d46528ba44f3cad4c5095/users/e1c23b5f03f6445e9a10b93fb775c60c/roles/3771341b3be641139e752aab690397a4 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:51:03.001186 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a0b149d3-b795-41e6-a46c-f51a73a905bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ace643dc2641461ead9b135ba23087aa'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:03.011889 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbb7118-7981-4dd3-b2f8-8d645688b367 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:03.012166 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbb7118-7981-4dd3-b2f8-8d645688b367 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:03.012479 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbb7118-7981-4dd3-b2f8-8d645688b367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:03.019624 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:51:03.051604 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-efbb7118-7981-4dd3-b2f8-8d645688b367 None None] MFA Rules not processed for user `e1c23b5f03f6445e9a10b93fb775c60c`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:51:03.053012 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-efbb7118-7981-4dd3-b2f8-8d645688b367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:51:03.081002 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1289/2612] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:51:02 2022] DELETE /identity/v3/users/ace643dc2641461ead9b135ba23087aa => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:51:03.128640 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1324/2613] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:51:03 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:51:03.142051 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b149d3-b795-41e6-a46c-f51a73a905bb admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:03.143495 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-a0b149d3-b795-41e6-a46c-f51a73a905bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:51:03.149435 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:51:03.204184 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b149d3-b795-41e6-a46c-f51a73a905bb admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:03.207778 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b1e2e1-348c-42d8-be16-4aa59d0e0fe7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b51da9867d9147f198b8077e953c4f9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:03.208476 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b1e2e1-348c-42d8-be16-4aa59d0e0fe7 service glance] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:03.208757 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b1e2e1-348c-42d8-be16-4aa59d0e0fe7 service glance] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:03.209033 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b1e2e1-348c-42d8-be16-4aa59d0e0fe7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:03.269475 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b1e2e1-348c-42d8-be16-4aa59d0e0fe7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:03.270975 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b1e2e1-348c-42d8-be16-4aa59d0e0fe7 service glance] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:03.283066 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1290/2614] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:51:03 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:51:03.563270 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbb7118-7981-4dd3-b2f8-8d645688b367 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:03.565015 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-efbb7118-7981-4dd3-b2f8-8d645688b367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:51:03.644651 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbb7118-7981-4dd3-b2f8-8d645688b367 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:03.706640 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b83472-b210-4783-81d3-7605ab9238e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:03.707502 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b83472-b210-4783-81d3-7605ab9238e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:03.707784 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b83472-b210-4783-81d3-7605ab9238e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:03.708062 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b83472-b210-4783-81d3-7605ab9238e8 admin admin] PATH_INFO: `/v3/projects/d008d0de59064c14b83275efaadb3c0e` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:03.709292 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b83472-b210-4783-81d3-7605ab9238e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d008d0de59064c14b83275efaadb3c0e)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:03.711771 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b83472-b210-4783-81d3-7605ab9238e8 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:03.860252 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:51:03.864059 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-e3b83472-b210-4783-81d3-7605ab9238e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd008d0de59064c14b83275efaadb3c0e', 'request_id': 'req-e3b83472-b210-4783-81d3-7605ab9238e8'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:03.966742 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-e3b83472-b210-4783-81d3-7605ab9238e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd008d0de59064c14b83275efaadb3c0e', 'request_id': 'req-e3b83472-b210-4783-81d3-7605ab9238e8'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:03.974226 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-e3b83472-b210-4783-81d3-7605ab9238e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd008d0de59064c14b83275efaadb3c0e', 'request_id': 'req-e3b83472-b210-4783-81d3-7605ab9238e8'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:03.985578 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-e3b83472-b210-4783-81d3-7605ab9238e8 admin admin] The token cache is being invalidate because project d008d0de59064c14b83275efaadb3c0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:51:03.985888 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-e3b83472-b210-4783-81d3-7605ab9238e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:03.986523 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-e3b83472-b210-4783-81d3-7605ab9238e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:03.988071 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1325/2615] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:51:03 2022] DELETE /identity/v3/projects/d008d0de59064c14b83275efaadb3c0e => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:51:03.993840 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1082dac-1e8d-44ad-941e-073bf7cec3a8 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:03.994145 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1082dac-1e8d-44ad-941e-073bf7cec3a8 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:03.994440 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1082dac-1e8d-44ad-941e-073bf7cec3a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:04.013157 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-e1082dac-1e8d-44ad-941e-073bf7cec3a8 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:51:04.014397 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-e1082dac-1e8d-44ad-941e-073bf7cec3a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:51:04.075975 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1291/2616] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:51:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:51:04.085151 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3581d8-3be4-492d-9aa7-8a7f1de97bf1 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:04.085477 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3581d8-3be4-492d-9aa7-8a7f1de97bf1 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:04.089862 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3581d8-3be4-492d-9aa7-8a7f1de97bf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:04.112728 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-4e3581d8-3be4-492d-9aa7-8a7f1de97bf1 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:51:04.113979 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-4e3581d8-3be4-492d-9aa7-8a7f1de97bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:51:04.122492 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1326/2617] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:51:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:51:04.130848 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1082dac-1e8d-44ad-941e-073bf7cec3a8 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:04.141463 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1082dac-1e8d-44ad-941e-073bf7cec3a8 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:04.151747 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d139f8a6-85ea-4614-b100-83db3952de66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:04.152457 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d139f8a6-85ea-4614-b100-83db3952de66 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:04.152748 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d139f8a6-85ea-4614-b100-83db3952de66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:04.153030 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d139f8a6-85ea-4614-b100-83db3952de66 admin admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:04.153796 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d139f8a6-85ea-4614-b100-83db3952de66 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:51:04.154133 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d139f8a6-85ea-4614-b100-83db3952de66 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:04.155734 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d139f8a6-85ea-4614-b100-83db3952de66 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:04.168717 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1292/2618] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jun 7 14:51:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:51:04.173245 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3581d8-3be4-492d-9aa7-8a7f1de97bf1 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:04.182969 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3581d8-3be4-492d-9aa7-8a7f1de97bf1 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:04.187760 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf090d6d-b101-4f68-a5ed-bd07ddc20dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:04.188445 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf090d6d-b101-4f68-a5ed-bd07ddc20dc9 admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:04.188732 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf090d6d-b101-4f68-a5ed-bd07ddc20dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:04.189091 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf090d6d-b101-4f68-a5ed-bd07ddc20dc9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:04.189857 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf090d6d-b101-4f68-a5ed-bd07ddc20dc9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:04.191753 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf090d6d-b101-4f68-a5ed-bd07ddc20dc9 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:04.280937 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1327/2619] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:51:04 2022] POST /identity/v3/projects => generated 383 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:51:04.285107 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d139f8a6-85ea-4614-b100-83db3952de66 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:04.295457 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d139f8a6-85ea-4614-b100-83db3952de66 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:04.337838 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:51:04.356900 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f21add7-1fbc-4e0b-afe6-97b6dee4e30a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:04.357598 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f21add7-1fbc-4e0b-afe6-97b6dee4e30a admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:04.357892 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f21add7-1fbc-4e0b-afe6-97b6dee4e30a admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:04.358170 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f21add7-1fbc-4e0b-afe6-97b6dee4e30a admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:04.358887 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f21add7-1fbc-4e0b-afe6-97b6dee4e30a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:04.360884 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f21add7-1fbc-4e0b-afe6-97b6dee4e30a admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:04.439225 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1293/2620] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:51:04 2022] POST /identity/v3/users => generated 292 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:51:04.443369 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf090d6d-b101-4f68-a5ed-bd07ddc20dc9 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:04.456128 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf090d6d-b101-4f68-a5ed-bd07ddc20dc9 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:04.463307 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d1ff46-9eb8-43dc-998e-eded57ac989a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:04.464402 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d1ff46-9eb8-43dc-998e-eded57ac989a admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:04.464698 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d1ff46-9eb8-43dc-998e-eded57ac989a admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:04.465041 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d1ff46-9eb8-43dc-998e-eded57ac989a admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:04.465778 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d1ff46-9eb8-43dc-998e-eded57ac989a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:04.467433 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d1ff46-9eb8-43dc-998e-eded57ac989a admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:04.478482 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1328/2621] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:51:04 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:51:04.482604 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f21add7-1fbc-4e0b-afe6-97b6dee4e30a admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:04.492857 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f21add7-1fbc-4e0b-afe6-97b6dee4e30a admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:04.497518 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2ddc60-c073-4bc8-b908-f3d18a53a819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:04.498309 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ddc60-c073-4bc8-b908-f3d18a53a819 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:04.498488 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ddc60-c073-4bc8-b908-f3d18a53a819 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:04.498770 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ddc60-c073-4bc8-b908-f3d18a53a819 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:04.499490 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2ddc60-c073-4bc8-b908-f3d18a53a819 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:04.501112 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2ddc60-c073-4bc8-b908-f3d18a53a819 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:04.516207 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1294/2622] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:51:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:51:04.520311 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d1ff46-9eb8-43dc-998e-eded57ac989a admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:04.531265 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d1ff46-9eb8-43dc-998e-eded57ac989a admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:04.536169 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e2f34e-3a9a-4a3f-a598-3bfda7b8b14a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:04.536933 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e2f34e-3a9a-4a3f-a598-3bfda7b8b14a admin admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:04.537221 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e2f34e-3a9a-4a3f-a598-3bfda7b8b14a admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:04.537505 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e2f34e-3a9a-4a3f-a598-3bfda7b8b14a admin admin] PATH_INFO: `/v3/projects/a53d22ab92d1449f81b9507734e9de68/users/b0e94b025ef0431a86e2b1f1a24d253c/roles/3771341b3be641139e752aab690397a4` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:04.554548 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e2f34e-3a9a-4a3f-a598-3bfda7b8b14a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a53d22ab92d1449f81b9507734e9de68, user_id=b0e94b025ef0431a86e2b1f1a24d253c, role_id=3771341b3be641139e752aab690397a4)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:04.557458 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e2f34e-3a9a-4a3f-a598-3bfda7b8b14a admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:04.645028 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1329/2623] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:51:04 2022] PUT /identity/v3/projects/a53d22ab92d1449f81b9507734e9de68/users/b0e94b025ef0431a86e2b1f1a24d253c/roles/3771341b3be641139e752aab690397a4 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:51:04.656392 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbc50ac-f7b6-48ec-80e3-e176a79afb1e None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:04.656698 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbc50ac-f7b6-48ec-80e3-e176a79afb1e None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:04.656986 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbc50ac-f7b6-48ec-80e3-e176a79afb1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:04.686240 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-acbc50ac-f7b6-48ec-80e3-e176a79afb1e None None] MFA Rules not processed for user `b0e94b025ef0431a86e2b1f1a24d253c`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:51:04.687490 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-acbc50ac-f7b6-48ec-80e3-e176a79afb1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:51:04.767097 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1295/2624] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:51:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:51:04.771713 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e2f34e-3a9a-4a3f-a598-3bfda7b8b14a admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:04.841210 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e2f34e-3a9a-4a3f-a598-3bfda7b8b14a admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:04.846121 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c27e68d-c5a4-4155-8db6-064928beed6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:04.846884 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c27e68d-c5a4-4155-8db6-064928beed6c admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:04.847178 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c27e68d-c5a4-4155-8db6-064928beed6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:04.847479 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c27e68d-c5a4-4155-8db6-064928beed6c admin admin] PATH_INFO: `/v3/projects` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:04.848163 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c27e68d-c5a4-4155-8db6-064928beed6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:04.850108 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c27e68d-c5a4-4155-8db6-064928beed6c admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:04.956954 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1330/2625] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:51:04 2022] POST /identity/v3/projects => generated 383 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:51:04.961695 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbc50ac-f7b6-48ec-80e3-e176a79afb1e None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:04.972533 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbc50ac-f7b6-48ec-80e3-e176a79afb1e None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:05.030837 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dcdd63-d9df-4d5a-8859-aa9f91b3ea54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:05.031545 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dcdd63-d9df-4d5a-8859-aa9f91b3ea54 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:05.031944 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dcdd63-d9df-4d5a-8859-aa9f91b3ea54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:05.032230 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dcdd63-d9df-4d5a-8859-aa9f91b3ea54 admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:05.032963 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dcdd63-d9df-4d5a-8859-aa9f91b3ea54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:05.035012 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dcdd63-d9df-4d5a-8859-aa9f91b3ea54 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:05.131768 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1296/2626] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:51:04 2022] POST /identity/v3/users => generated 298 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:51:05.135949 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c27e68d-c5a4-4155-8db6-064928beed6c admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:05.147280 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c27e68d-c5a4-4155-8db6-064928beed6c admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:05.152305 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815aebe1-2f16-45a8-9ab9-9b41cbf952b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:05.153009 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815aebe1-2f16-45a8-9ab9-9b41cbf952b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:05.153337 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815aebe1-2f16-45a8-9ab9-9b41cbf952b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:05.153645 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815aebe1-2f16-45a8-9ab9-9b41cbf952b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:05.154379 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815aebe1-2f16-45a8-9ab9-9b41cbf952b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:05.155985 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815aebe1-2f16-45a8-9ab9-9b41cbf952b7 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:05.172962 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1331/2627] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:51:05 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 14:51:05.177457 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dcdd63-d9df-4d5a-8859-aa9f91b3ea54 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:05.189184 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dcdd63-d9df-4d5a-8859-aa9f91b3ea54 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:05.194130 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aad727-dd14-4876-9831-27a99dfc1885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:05.198695 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aad727-dd14-4876-9831-27a99dfc1885 admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:05.198695 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aad727-dd14-4876-9831-27a99dfc1885 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:05.198695 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aad727-dd14-4876-9831-27a99dfc1885 admin admin] PATH_INFO: `/v3/projects/f4e01d8728a64da79a9e08d12feaa57d/users/967bc3f911834bbbba98e6d3891f8132/roles/9baff2b18f2649f7b757ff2e1f63c853` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:05.213927 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aad727-dd14-4876-9831-27a99dfc1885 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4e01d8728a64da79a9e08d12feaa57d, user_id=967bc3f911834bbbba98e6d3891f8132, role_id=9baff2b18f2649f7b757ff2e1f63c853)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:05.216688 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aad727-dd14-4876-9831-27a99dfc1885 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:05.286934 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1297/2628] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:51:05 2022] PUT /identity/v3/projects/f4e01d8728a64da79a9e08d12feaa57d/users/967bc3f911834bbbba98e6d3891f8132/roles/9baff2b18f2649f7b757ff2e1f63c853 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:51:05.291514 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815aebe1-2f16-45a8-9ab9-9b41cbf952b7 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:05.302509 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815aebe1-2f16-45a8-9ab9-9b41cbf952b7 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:05.343445 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:51:05.361285 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4689c34c-9126-47e8-9d63-d0eacd111873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:05.361961 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4689c34c-9126-47e8-9d63-d0eacd111873 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:05.362247 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4689c34c-9126-47e8-9d63-d0eacd111873 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:05.362525 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4689c34c-9126-47e8-9d63-d0eacd111873 admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:05.363261 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4689c34c-9126-47e8-9d63-d0eacd111873 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:05.364937 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4689c34c-9126-47e8-9d63-d0eacd111873 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:05.380623 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1332/2629] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:51:05 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:51:05.381887 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aad727-dd14-4876-9831-27a99dfc1885 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:05.392803 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aad727-dd14-4876-9831-27a99dfc1885 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:05.398076 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8775b47d-1edc-4874-a6b8-2e4d07d778d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:05.399647 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8775b47d-1edc-4874-a6b8-2e4d07d778d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:05.399938 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8775b47d-1edc-4874-a6b8-2e4d07d778d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:05.400220 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8775b47d-1edc-4874-a6b8-2e4d07d778d7 admin admin] PATH_INFO: `/v3/projects/f4e01d8728a64da79a9e08d12feaa57d/users/967bc3f911834bbbba98e6d3891f8132/roles/9baff2b18f2649f7b757ff2e1f63c853` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:05.402553 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8775b47d-1edc-4874-a6b8-2e4d07d778d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4e01d8728a64da79a9e08d12feaa57d, user_id=967bc3f911834bbbba98e6d3891f8132, role_id=9baff2b18f2649f7b757ff2e1f63c853)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:05.405316 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8775b47d-1edc-4874-a6b8-2e4d07d778d7 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:05.416681 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1298/2630] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:51:05 2022] PUT /identity/v3/projects/f4e01d8728a64da79a9e08d12feaa57d/users/967bc3f911834bbbba98e6d3891f8132/roles/9baff2b18f2649f7b757ff2e1f63c853 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:51:05.428080 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34688ff8-ef8d-4e49-91a7-4be3dbcf788c None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:05.428392 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34688ff8-ef8d-4e49-91a7-4be3dbcf788c None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:05.428693 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34688ff8-ef8d-4e49-91a7-4be3dbcf788c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:05.456321 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-34688ff8-ef8d-4e49-91a7-4be3dbcf788c None None] MFA Rules not processed for user `967bc3f911834bbbba98e6d3891f8132`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:51:05.457770 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-34688ff8-ef8d-4e49-91a7-4be3dbcf788c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:51:05.525656 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1333/2631] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:51:05 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:51:05.538270 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8775b47d-1edc-4874-a6b8-2e4d07d778d7 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:05.540166 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-8775b47d-1edc-4874-a6b8-2e4d07d778d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:51:05.546956 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:51:05.608067 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8775b47d-1edc-4874-a6b8-2e4d07d778d7 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:05.611762 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251ce909-18a6-43e6-a5ac-8b1319e9d563 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d81f597cb44c56a966537089bcce4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:05.612475 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251ce909-18a6-43e6-a5ac-8b1319e9d563 service cinder] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:05.612757 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251ce909-18a6-43e6-a5ac-8b1319e9d563 service cinder] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:05.613036 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251ce909-18a6-43e6-a5ac-8b1319e9d563 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:05.624159 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251ce909-18a6-43e6-a5ac-8b1319e9d563 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:05.625626 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251ce909-18a6-43e6-a5ac-8b1319e9d563 service cinder] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:05.636413 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1299/2632] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:51:05 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:51:05.763647 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34688ff8-ef8d-4e49-91a7-4be3dbcf788c None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:05.773742 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34688ff8-ef8d-4e49-91a7-4be3dbcf788c None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:05.778231 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcfa6b5-2526-41a1-9331-d16b99429cbc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d81f597cb44c56a966537089bcce4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:05.779051 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcfa6b5-2526-41a1-9331-d16b99429cbc service cinder] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:05.779336 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcfa6b5-2526-41a1-9331-d16b99429cbc service cinder] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:05.779683 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcfa6b5-2526-41a1-9331-d16b99429cbc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:05.839655 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcfa6b5-2526-41a1-9331-d16b99429cbc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:05.841104 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcfa6b5-2526-41a1-9331-d16b99429cbc service cinder] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:05.849965 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1334/2633] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:51:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:51:06.347384 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251ce909-18a6-43e6-a5ac-8b1319e9d563 service cinder] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:06.415593 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251ce909-18a6-43e6-a5ac-8b1319e9d563 service cinder] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:06.420188 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f71717-8b85-4374-81b3-7d491272e695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:06.421180 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f71717-8b85-4374-81b3-7d491272e695 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:06.421474 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f71717-8b85-4374-81b3-7d491272e695 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:06.421755 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f71717-8b85-4374-81b3-7d491272e695 admin admin] PATH_INFO: `/v3/users/b0e94b025ef0431a86e2b1f1a24d253c` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:06.422959 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f71717-8b85-4374-81b3-7d491272e695 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0e94b025ef0431a86e2b1f1a24d253c)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:06.424909 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f71717-8b85-4374-81b3-7d491272e695 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:06.602628 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-b6f71717-8b85-4374-81b3-7d491272e695 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0e94b025ef0431a86e2b1f1a24d253c'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:06.603317 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-b6f71717-8b85-4374-81b3-7d491272e695 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0e94b025ef0431a86e2b1f1a24d253c'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:06.603997 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-b6f71717-8b85-4374-81b3-7d491272e695 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0e94b025ef0431a86e2b1f1a24d253c'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:06.611196 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-b6f71717-8b85-4374-81b3-7d491272e695 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0e94b025ef0431a86e2b1f1a24d253c'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:06.624223 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-b6f71717-8b85-4374-81b3-7d491272e695 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0e94b025ef0431a86e2b1f1a24d253c'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:06.703625 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1300/2634] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:51:06 2022] DELETE /identity/v3/users/b0e94b025ef0431a86e2b1f1a24d253c => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:51:06.708429 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbcd9eb-0ba8-4e06-b2cc-a9c1c2548ce0 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:06.708729 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbcd9eb-0ba8-4e06-b2cc-a9c1c2548ce0 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:06.709017 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbcd9eb-0ba8-4e06-b2cc-a9c1c2548ce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:06.728366 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-5cbcd9eb-0ba8-4e06-b2cc-a9c1c2548ce0 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:51:06.729696 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-5cbcd9eb-0ba8-4e06-b2cc-a9c1c2548ce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:51:06.791105 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1335/2635] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:51:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:51:06.805758 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f71717-8b85-4374-81b3-7d491272e695 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:06.807454 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-b6f71717-8b85-4374-81b3-7d491272e695 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:51:06.841133 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:51:06.866963 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f71717-8b85-4374-81b3-7d491272e695 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:06.870614 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e363c9-da4b-4ec1-9599-6854d0867302 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:06.871344 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e363c9-da4b-4ec1-9599-6854d0867302 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:06.871628 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e363c9-da4b-4ec1-9599-6854d0867302 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:06.871930 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e363c9-da4b-4ec1-9599-6854d0867302 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:06.882876 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e363c9-da4b-4ec1-9599-6854d0867302 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:06.889227 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e363c9-da4b-4ec1-9599-6854d0867302 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:06.895082 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1301/2636] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:51:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:51:07.312717 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbcd9eb-0ba8-4e06-b2cc-a9c1c2548ce0 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:07.314175 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-5cbcd9eb-0ba8-4e06-b2cc-a9c1c2548ce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:51:07.321203 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=100 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:51:07.328334 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbcd9eb-0ba8-4e06-b2cc-a9c1c2548ce0 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:07.334273 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeac528-71e7-447f-9d34-a1e4fc02e8de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:07.335220 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeac528-71e7-447f-9d34-a1e4fc02e8de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:07.335505 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeac528-71e7-447f-9d34-a1e4fc02e8de admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:07.335783 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeac528-71e7-447f-9d34-a1e4fc02e8de admin admin] PATH_INFO: `/v3/projects/a53d22ab92d1449f81b9507734e9de68` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:07.338140 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeac528-71e7-447f-9d34-a1e4fc02e8de admin admin] RBAC: Authorizing `identity:delete_project(project_id=a53d22ab92d1449f81b9507734e9de68)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:07.339139 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeac528-71e7-447f-9d34-a1e4fc02e8de admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:07.477826 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-0eeac528-71e7-447f-9d34-a1e4fc02e8de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a53d22ab92d1449f81b9507734e9de68', 'request_id': 'req-0eeac528-71e7-447f-9d34-a1e4fc02e8de'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:07.549238 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-0eeac528-71e7-447f-9d34-a1e4fc02e8de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a53d22ab92d1449f81b9507734e9de68', 'request_id': 'req-0eeac528-71e7-447f-9d34-a1e4fc02e8de'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:07.555842 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-0eeac528-71e7-447f-9d34-a1e4fc02e8de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a53d22ab92d1449f81b9507734e9de68', 'request_id': 'req-0eeac528-71e7-447f-9d34-a1e4fc02e8de'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:07.567698 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-0eeac528-71e7-447f-9d34-a1e4fc02e8de admin admin] The token cache is being invalidate because project a53d22ab92d1449f81b9507734e9de68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:51:07.567990 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-0eeac528-71e7-447f-9d34-a1e4fc02e8de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:07.568659 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-0eeac528-71e7-447f-9d34-a1e4fc02e8de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:07.570189 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1336/2637] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:51:07 2022] DELETE /identity/v3/projects/a53d22ab92d1449f81b9507734e9de68 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:51:07.574166 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e363c9-da4b-4ec1-9599-6854d0867302 service neutron] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:07.583745 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e363c9-da4b-4ec1-9599-6854d0867302 service neutron] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:07.651944 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bc00de-60de-4119-8cd1-8e7067612ee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:07.652920 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bc00de-60de-4119-8cd1-8e7067612ee0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:07.653227 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bc00de-60de-4119-8cd1-8e7067612ee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:07.653506 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bc00de-60de-4119-8cd1-8e7067612ee0 admin admin] PATH_INFO: `/v3/users/967bc3f911834bbbba98e6d3891f8132` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:07.654720 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bc00de-60de-4119-8cd1-8e7067612ee0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=967bc3f911834bbbba98e6d3891f8132)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:07.658009 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bc00de-60de-4119-8cd1-8e7067612ee0 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:07.896058 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-40bc00de-60de-4119-8cd1-8e7067612ee0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '967bc3f911834bbbba98e6d3891f8132'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:07.896760 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-40bc00de-60de-4119-8cd1-8e7067612ee0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '967bc3f911834bbbba98e6d3891f8132'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:07.897371 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-40bc00de-60de-4119-8cd1-8e7067612ee0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '967bc3f911834bbbba98e6d3891f8132'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:07.905398 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-40bc00de-60de-4119-8cd1-8e7067612ee0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '967bc3f911834bbbba98e6d3891f8132'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:07.918245 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-40bc00de-60de-4119-8cd1-8e7067612ee0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '967bc3f911834bbbba98e6d3891f8132'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:07.983732 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1302/2638] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:51:07 2022] DELETE /identity/v3/users/967bc3f911834bbbba98e6d3891f8132 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:51:08.396435 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeac528-71e7-447f-9d34-a1e4fc02e8de admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:08.397954 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-0eeac528-71e7-447f-9d34-a1e4fc02e8de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:51:08.416547 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeac528-71e7-447f-9d34-a1e4fc02e8de admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:08.486292 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebeada6-5f60-4fdd-a3b9-6666c308fd28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:08.487244 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebeada6-5f60-4fdd-a3b9-6666c308fd28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:08.489815 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebeada6-5f60-4fdd-a3b9-6666c308fd28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:08.489815 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebeada6-5f60-4fdd-a3b9-6666c308fd28 admin admin] PATH_INFO: `/v3/projects/f4e01d8728a64da79a9e08d12feaa57d` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:08.489815 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebeada6-5f60-4fdd-a3b9-6666c308fd28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4e01d8728a64da79a9e08d12feaa57d)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:08.491080 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebeada6-5f60-4fdd-a3b9-6666c308fd28 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:08.641671 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-7ebeada6-5f60-4fdd-a3b9-6666c308fd28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4e01d8728a64da79a9e08d12feaa57d', 'request_id': 'req-7ebeada6-5f60-4fdd-a3b9-6666c308fd28'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:08.749485 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-7ebeada6-5f60-4fdd-a3b9-6666c308fd28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4e01d8728a64da79a9e08d12feaa57d', 'request_id': 'req-7ebeada6-5f60-4fdd-a3b9-6666c308fd28'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:08.755731 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-7ebeada6-5f60-4fdd-a3b9-6666c308fd28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4e01d8728a64da79a9e08d12feaa57d', 'request_id': 'req-7ebeada6-5f60-4fdd-a3b9-6666c308fd28'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:08.768159 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-7ebeada6-5f60-4fdd-a3b9-6666c308fd28 admin admin] The token cache is being invalidate because project f4e01d8728a64da79a9e08d12feaa57d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:51:08.768585 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-7ebeada6-5f60-4fdd-a3b9-6666c308fd28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:08.769234 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-7ebeada6-5f60-4fdd-a3b9-6666c308fd28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:08.770883 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1337/2639] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:51:08 2022] DELETE /identity/v3/projects/f4e01d8728a64da79a9e08d12feaa57d => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:51:08.777269 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4775961-5c39-4cd6-a7b0-5b71a6d7cae8 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:08.777837 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4775961-5c39-4cd6-a7b0-5b71a6d7cae8 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:08.777837 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4775961-5c39-4cd6-a7b0-5b71a6d7cae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:08.796141 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-c4775961-5c39-4cd6-a7b0-5b71a6d7cae8 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:51:08.797382 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-c4775961-5c39-4cd6-a7b0-5b71a6d7cae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:51:08.842289 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:51:08.858268 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1303/2640] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:51:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:51:08.866440 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43edc397-6507-417b-84d0-45838caa1435 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:08.866731 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43edc397-6507-417b-84d0-45838caa1435 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:08.867019 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43edc397-6507-417b-84d0-45838caa1435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:08.887735 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-43edc397-6507-417b-84d0-45838caa1435 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:51:08.889658 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-43edc397-6507-417b-84d0-45838caa1435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:51:08.899280 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1338/2641] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:51:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:51:08.903821 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4775961-5c39-4cd6-a7b0-5b71a6d7cae8 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:08.915848 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4775961-5c39-4cd6-a7b0-5b71a6d7cae8 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:08.920650 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24aa943-4f34-4fe6-95db-6f938b4670df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:08.921309 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24aa943-4f34-4fe6-95db-6f938b4670df admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:08.921592 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24aa943-4f34-4fe6-95db-6f938b4670df admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:08.921868 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24aa943-4f34-4fe6-95db-6f938b4670df admin admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:08.922870 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24aa943-4f34-4fe6-95db-6f938b4670df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:51:08.923080 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24aa943-4f34-4fe6-95db-6f938b4670df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:08.925077 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24aa943-4f34-4fe6-95db-6f938b4670df admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:08.937789 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1304/2642] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jun 7 14:51:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:51:08.942768 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43edc397-6507-417b-84d0-45838caa1435 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:08.952565 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43edc397-6507-417b-84d0-45838caa1435 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:08.957682 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1f54f2-d3c3-4557-b5e1-cb6016295894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:08.958452 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1f54f2-d3c3-4557-b5e1-cb6016295894 admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:08.958966 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1f54f2-d3c3-4557-b5e1-cb6016295894 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:08.959370 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1f54f2-d3c3-4557-b5e1-cb6016295894 admin admin] PATH_INFO: `/v3/projects` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:08.960134 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1f54f2-d3c3-4557-b5e1-cb6016295894 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:08.962511 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1f54f2-d3c3-4557-b5e1-cb6016295894 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:08.966767 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:51:09.082550 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1339/2643] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:51:08 2022] POST /identity/v3/projects => generated 385 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:51:09.087208 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24aa943-4f34-4fe6-95db-6f938b4670df admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:09.097325 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24aa943-4f34-4fe6-95db-6f938b4670df admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:09.156042 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2929916c-49c8-43ef-8747-78c34841d676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:09.156781 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2929916c-49c8-43ef-8747-78c34841d676 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:09.157067 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2929916c-49c8-43ef-8747-78c34841d676 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:09.157339 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2929916c-49c8-43ef-8747-78c34841d676 admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:09.158031 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2929916c-49c8-43ef-8747-78c34841d676 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:09.160009 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2929916c-49c8-43ef-8747-78c34841d676 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:09.252970 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1305/2644] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:51:09 2022] POST /identity/v3/users => generated 293 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:51:09.257399 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1f54f2-d3c3-4557-b5e1-cb6016295894 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:09.268000 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1f54f2-d3c3-4557-b5e1-cb6016295894 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:09.273069 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76c7590-aa7d-445b-adcb-9c521ca5a5bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:09.274248 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76c7590-aa7d-445b-adcb-9c521ca5a5bf admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:09.274533 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76c7590-aa7d-445b-adcb-9c521ca5a5bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:09.274818 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76c7590-aa7d-445b-adcb-9c521ca5a5bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:09.275582 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76c7590-aa7d-445b-adcb-9c521ca5a5bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:09.277721 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76c7590-aa7d-445b-adcb-9c521ca5a5bf admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:09.290343 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1340/2645] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:51:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:51:09.294397 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2929916c-49c8-43ef-8747-78c34841d676 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:09.307572 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2929916c-49c8-43ef-8747-78c34841d676 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:09.309534 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a3aedc-196d-4864-b30a-940c780cb11e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:09.310198 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a3aedc-196d-4864-b30a-940c780cb11e admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:09.310492 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a3aedc-196d-4864-b30a-940c780cb11e admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:09.311068 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a3aedc-196d-4864-b30a-940c780cb11e admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:09.311496 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a3aedc-196d-4864-b30a-940c780cb11e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:09.313172 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a3aedc-196d-4864-b30a-940c780cb11e admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:09.327403 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1306/2646] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:51:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:51:09.328661 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76c7590-aa7d-445b-adcb-9c521ca5a5bf admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:09.339936 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76c7590-aa7d-445b-adcb-9c521ca5a5bf admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:09.366476 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b803e5-1859-421e-a6e5-034118d50192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:09.367650 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b803e5-1859-421e-a6e5-034118d50192 admin admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:09.368022 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b803e5-1859-421e-a6e5-034118d50192 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:09.368311 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b803e5-1859-421e-a6e5-034118d50192 admin admin] PATH_INFO: `/v3/projects/27f784d297fb44d8a214cc1c2f3a95fa/users/4b3d981122024e02ab51cc8855724525/roles/3771341b3be641139e752aab690397a4` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:09.387204 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b803e5-1859-421e-a6e5-034118d50192 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27f784d297fb44d8a214cc1c2f3a95fa, user_id=4b3d981122024e02ab51cc8855724525, role_id=3771341b3be641139e752aab690397a4)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:09.389929 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b803e5-1859-421e-a6e5-034118d50192 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:09.476045 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1341/2647] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:51:09 2022] PUT /identity/v3/projects/27f784d297fb44d8a214cc1c2f3a95fa/users/4b3d981122024e02ab51cc8855724525/roles/3771341b3be641139e752aab690397a4 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:51:09.487178 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d71a0-c602-4387-9af0-9b57232d588f None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:09.487472 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d71a0-c602-4387-9af0-9b57232d588f None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:09.489937 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d71a0-c602-4387-9af0-9b57232d588f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:09.515881 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-ae4d71a0-c602-4387-9af0-9b57232d588f None None] MFA Rules not processed for user `4b3d981122024e02ab51cc8855724525`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:51:09.517139 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-ae4d71a0-c602-4387-9af0-9b57232d588f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:51:09.583821 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1307/2648] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:51:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:51:09.589300 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b803e5-1859-421e-a6e5-034118d50192 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:09.664953 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b803e5-1859-421e-a6e5-034118d50192 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:09.669608 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91923065-d973-4ef7-85e5-01f017f8f941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:09.670276 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91923065-d973-4ef7-85e5-01f017f8f941 admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:09.670600 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91923065-d973-4ef7-85e5-01f017f8f941 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:09.670842 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91923065-d973-4ef7-85e5-01f017f8f941 admin admin] PATH_INFO: `/v3/projects` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:09.671546 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91923065-d973-4ef7-85e5-01f017f8f941 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:09.673481 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91923065-d973-4ef7-85e5-01f017f8f941 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:09.798140 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1342/2649] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:51:09 2022] POST /identity/v3/projects => generated 383 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:51:09.802572 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4d71a0-c602-4387-9af0-9b57232d588f None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:09.815511 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4d71a0-c602-4387-9af0-9b57232d588f None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:09.860227 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:51:09.875322 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a826d5-695f-4ed7-9d20-1d72cbc69efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:09.876029 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a826d5-695f-4ed7-9d20-1d72cbc69efe admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:09.876318 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a826d5-695f-4ed7-9d20-1d72cbc69efe admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:09.876632 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a826d5-695f-4ed7-9d20-1d72cbc69efe admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:09.877324 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a826d5-695f-4ed7-9d20-1d72cbc69efe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:09.879266 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a826d5-695f-4ed7-9d20-1d72cbc69efe admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:09.962813 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1308/2650] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:51:09 2022] POST /identity/v3/users => generated 292 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:51:09.965420 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91923065-d973-4ef7-85e5-01f017f8f941 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:09.976146 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91923065-d973-4ef7-85e5-01f017f8f941 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:09.985252 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ad78f1-baa3-4b76-8ce2-cf4964aef633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:09.985252 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ad78f1-baa3-4b76-8ce2-cf4964aef633 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:09.985252 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ad78f1-baa3-4b76-8ce2-cf4964aef633 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:09.985252 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ad78f1-baa3-4b76-8ce2-cf4964aef633 admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:09.985704 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ad78f1-baa3-4b76-8ce2-cf4964aef633 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:09.985704 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ad78f1-baa3-4b76-8ce2-cf4964aef633 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:09.997572 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1343/2651] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:51:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:51:10.001573 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a826d5-695f-4ed7-9d20-1d72cbc69efe admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:10.012566 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a826d5-695f-4ed7-9d20-1d72cbc69efe admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:10.017298 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db58bd3-5acd-41b2-bdbb-c13632e9e132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:10.018233 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db58bd3-5acd-41b2-bdbb-c13632e9e132 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:10.018233 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db58bd3-5acd-41b2-bdbb-c13632e9e132 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:10.018531 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db58bd3-5acd-41b2-bdbb-c13632e9e132 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:10.019298 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db58bd3-5acd-41b2-bdbb-c13632e9e132 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:10.020966 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db58bd3-5acd-41b2-bdbb-c13632e9e132 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:10.033245 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1309/2652] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:51:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:51:10.038345 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ad78f1-baa3-4b76-8ce2-cf4964aef633 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:10.048900 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ad78f1-baa3-4b76-8ce2-cf4964aef633 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:10.053623 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00c997c-d620-4d6f-bc19-acfa7a565288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:10.054521 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00c997c-d620-4d6f-bc19-acfa7a565288 admin admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:10.055323 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00c997c-d620-4d6f-bc19-acfa7a565288 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:10.055596 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00c997c-d620-4d6f-bc19-acfa7a565288 admin admin] PATH_INFO: `/v3/projects/af31ae023fa940d9a65d9f9727cbe832/users/5867aa461249410a8451a7502f3b77ad/roles/3771341b3be641139e752aab690397a4` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:10.071782 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00c997c-d620-4d6f-bc19-acfa7a565288 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af31ae023fa940d9a65d9f9727cbe832, user_id=5867aa461249410a8451a7502f3b77ad, role_id=3771341b3be641139e752aab690397a4)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:10.074542 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00c997c-d620-4d6f-bc19-acfa7a565288 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:10.144914 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1344/2653] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:51:10 2022] PUT /identity/v3/projects/af31ae023fa940d9a65d9f9727cbe832/users/5867aa461249410a8451a7502f3b77ad/roles/3771341b3be641139e752aab690397a4 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:51:10.157746 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399f6c1d-26a2-4baa-8893-ac4cde65a285 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:10.158010 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399f6c1d-26a2-4baa-8893-ac4cde65a285 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:10.158257 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399f6c1d-26a2-4baa-8893-ac4cde65a285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:10.188365 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-399f6c1d-26a2-4baa-8893-ac4cde65a285 None None] MFA Rules not processed for user `5867aa461249410a8451a7502f3b77ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:51:10.189609 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-399f6c1d-26a2-4baa-8893-ac4cde65a285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:51:10.257183 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1310/2654] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:51:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:51:10.262395 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00c997c-d620-4d6f-bc19-acfa7a565288 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:10.326297 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00c997c-d620-4d6f-bc19-acfa7a565288 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:10.331192 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6794e604-25cc-4bc6-b011-cacad0dde074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:10.331901 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6794e604-25cc-4bc6-b011-cacad0dde074 admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:10.332228 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6794e604-25cc-4bc6-b011-cacad0dde074 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:10.332500 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6794e604-25cc-4bc6-b011-cacad0dde074 admin admin] PATH_INFO: `/v3/projects` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:10.333199 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6794e604-25cc-4bc6-b011-cacad0dde074 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:10.335105 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6794e604-25cc-4bc6-b011-cacad0dde074 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:10.339474 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:51:10.494840 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1345/2655] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:51:10 2022] POST /identity/v3/projects => generated 385 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:51:10.505191 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399f6c1d-26a2-4baa-8893-ac4cde65a285 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:10.516886 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399f6c1d-26a2-4baa-8893-ac4cde65a285 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:10.578897 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ac8771-3d44-4419-9d9d-f6a1c138684c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:10.579667 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ac8771-3d44-4419-9d9d-f6a1c138684c admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:10.580009 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ac8771-3d44-4419-9d9d-f6a1c138684c admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:10.580285 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ac8771-3d44-4419-9d9d-f6a1c138684c admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:10.581128 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ac8771-3d44-4419-9d9d-f6a1c138684c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:10.583114 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ac8771-3d44-4419-9d9d-f6a1c138684c admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:10.731927 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1311/2656] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:51:10 2022] POST /identity/v3/users => generated 299 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:51:10.735977 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6794e604-25cc-4bc6-b011-cacad0dde074 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:10.746943 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6794e604-25cc-4bc6-b011-cacad0dde074 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:10.751633 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e14f8f3-c77d-4957-91cc-a950ef019d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:10.752300 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e14f8f3-c77d-4957-91cc-a950ef019d72 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:10.752620 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e14f8f3-c77d-4957-91cc-a950ef019d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:10.752969 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e14f8f3-c77d-4957-91cc-a950ef019d72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:10.753656 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e14f8f3-c77d-4957-91cc-a950ef019d72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:10.755266 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e14f8f3-c77d-4957-91cc-a950ef019d72 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:10.787344 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1346/2657] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:51:10 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:51:10.793162 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ac8771-3d44-4419-9d9d-f6a1c138684c admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:10.806423 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ac8771-3d44-4419-9d9d-f6a1c138684c admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:10.811290 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d873e517-52c4-406a-b327-5707b97fc691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:10.812204 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d873e517-52c4-406a-b327-5707b97fc691 admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:10.812521 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d873e517-52c4-406a-b327-5707b97fc691 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:10.812804 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d873e517-52c4-406a-b327-5707b97fc691 admin admin] PATH_INFO: `/v3/projects/f883e059c87c4f7da6e8f5309e6a1b96/users/b803ec3c6a994b738c036223ac89adc3/roles/9baff2b18f2649f7b757ff2e1f63c853` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:10.829952 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d873e517-52c4-406a-b327-5707b97fc691 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f883e059c87c4f7da6e8f5309e6a1b96, user_id=b803ec3c6a994b738c036223ac89adc3, role_id=9baff2b18f2649f7b757ff2e1f63c853)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:10.836599 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d873e517-52c4-406a-b327-5707b97fc691 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:10.996108 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1312/2658] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:51:10 2022] PUT /identity/v3/projects/f883e059c87c4f7da6e8f5309e6a1b96/users/b803ec3c6a994b738c036223ac89adc3/roles/9baff2b18f2649f7b757ff2e1f63c853 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:51:11.000651 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e14f8f3-c77d-4957-91cc-a950ef019d72 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:11.010992 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e14f8f3-c77d-4957-91cc-a950ef019d72 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:11.074004 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217fd333-b4da-4f9b-9560-bb3a35277cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:11.074721 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217fd333-b4da-4f9b-9560-bb3a35277cc5 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:11.075015 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217fd333-b4da-4f9b-9560-bb3a35277cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:11.075301 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217fd333-b4da-4f9b-9560-bb3a35277cc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:11.076053 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217fd333-b4da-4f9b-9560-bb3a35277cc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:11.077746 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217fd333-b4da-4f9b-9560-bb3a35277cc5 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:11.088923 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1347/2659] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:51:10 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:51:11.093385 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d873e517-52c4-406a-b327-5707b97fc691 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:11.103937 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d873e517-52c4-406a-b327-5707b97fc691 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:11.108757 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cd1335-9647-4ee9-95d6-07c2660bcaab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:11.109608 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cd1335-9647-4ee9-95d6-07c2660bcaab admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:11.109909 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cd1335-9647-4ee9-95d6-07c2660bcaab admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:11.110210 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cd1335-9647-4ee9-95d6-07c2660bcaab admin admin] PATH_INFO: `/v3/projects/f883e059c87c4f7da6e8f5309e6a1b96/users/b803ec3c6a994b738c036223ac89adc3/roles/9baff2b18f2649f7b757ff2e1f63c853` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:11.113432 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cd1335-9647-4ee9-95d6-07c2660bcaab admin admin] RBAC: Authorizing `identity:create_grant(project_id=f883e059c87c4f7da6e8f5309e6a1b96, user_id=b803ec3c6a994b738c036223ac89adc3, role_id=9baff2b18f2649f7b757ff2e1f63c853)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:11.118679 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cd1335-9647-4ee9-95d6-07c2660bcaab admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:11.132079 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1313/2660] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:51:11 2022] PUT /identity/v3/projects/f883e059c87c4f7da6e8f5309e6a1b96/users/b803ec3c6a994b738c036223ac89adc3/roles/9baff2b18f2649f7b757ff2e1f63c853 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:51:11.142711 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58a5887-c59d-4a8f-a3f9-c6779630d36e None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:11.143076 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58a5887-c59d-4a8f-a3f9-c6779630d36e None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:11.149268 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58a5887-c59d-4a8f-a3f9-c6779630d36e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:11.172239 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-b58a5887-c59d-4a8f-a3f9-c6779630d36e None None] MFA Rules not processed for user `b803ec3c6a994b738c036223ac89adc3`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:51:11.173511 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-b58a5887-c59d-4a8f-a3f9-c6779630d36e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:51:11.243299 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1348/2661] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:51:11 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:51:11.255016 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cd1335-9647-4ee9-95d6-07c2660bcaab admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:11.256866 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-69cd1335-9647-4ee9-95d6-07c2660bcaab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:51:11.264387 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:51:11.319611 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cd1335-9647-4ee9-95d6-07c2660bcaab admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:11.324113 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fca1eef-41a2-4c93-91e2-974481d57052 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d81f597cb44c56a966537089bcce4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:11.324783 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fca1eef-41a2-4c93-91e2-974481d57052 service cinder] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:11.325093 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fca1eef-41a2-4c93-91e2-974481d57052 service cinder] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:11.325371 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fca1eef-41a2-4c93-91e2-974481d57052 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:11.387889 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fca1eef-41a2-4c93-91e2-974481d57052 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:11.389389 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fca1eef-41a2-4c93-91e2-974481d57052 service cinder] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:11.400200 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1314/2662] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:51:11 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:51:14.373010 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58a5887-c59d-4a8f-a3f9-c6779630d36e None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:14.383499 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58a5887-c59d-4a8f-a3f9-c6779630d36e None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:14.387261 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4de5b8-abfc-41c6-af52-13a37a4fe3a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d81f597cb44c56a966537089bcce4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:14.387915 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4de5b8-abfc-41c6-af52-13a37a4fe3a6 service cinder] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:14.388197 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4de5b8-abfc-41c6-af52-13a37a4fe3a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:14.388524 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4de5b8-abfc-41c6-af52-13a37a4fe3a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:14.449774 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:51:14.452024 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4de5b8-abfc-41c6-af52-13a37a4fe3a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:14.453491 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4de5b8-abfc-41c6-af52-13a37a4fe3a6 service cinder] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:14.462494 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1349/2663] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:51:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:51:15.821632 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fca1eef-41a2-4c93-91e2-974481d57052 service cinder] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:15.823107 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-9fca1eef-41a2-4c93-91e2-974481d57052 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:51:15.834710 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fca1eef-41a2-4c93-91e2-974481d57052 service cinder] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:15.838382 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f35c53-0bb4-41f4-8591-ddbaef2da5d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d81f597cb44c56a966537089bcce4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:15.839026 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f35c53-0bb4-41f4-8591-ddbaef2da5d7 service cinder] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:15.839313 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f35c53-0bb4-41f4-8591-ddbaef2da5d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:15.839596 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f35c53-0bb4-41f4-8591-ddbaef2da5d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:15.849311 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f35c53-0bb4-41f4-8591-ddbaef2da5d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:15.851356 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f35c53-0bb4-41f4-8591-ddbaef2da5d7 service cinder] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:15.861584 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1315/2664] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:51:15 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:51:21.641403 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4de5b8-abfc-41c6-af52-13a37a4fe3a6 service cinder] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:21.751756 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4de5b8-abfc-41c6-af52-13a37a4fe3a6 service cinder] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:21.756553 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5038a24-3bf3-4154-ab3c-711adfb8ca64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:21.757431 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5038a24-3bf3-4154-ab3c-711adfb8ca64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:21.757721 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5038a24-3bf3-4154-ab3c-711adfb8ca64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:21.758002 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5038a24-3bf3-4154-ab3c-711adfb8ca64 admin admin] PATH_INFO: `/v3/users/4b3d981122024e02ab51cc8855724525` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:21.759228 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5038a24-3bf3-4154-ab3c-711adfb8ca64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b3d981122024e02ab51cc8855724525)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:21.761236 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5038a24-3bf3-4154-ab3c-711adfb8ca64 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:21.922843 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-c5038a24-3bf3-4154-ab3c-711adfb8ca64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b3d981122024e02ab51cc8855724525'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:21.923420 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-c5038a24-3bf3-4154-ab3c-711adfb8ca64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b3d981122024e02ab51cc8855724525'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:21.924034 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-c5038a24-3bf3-4154-ab3c-711adfb8ca64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b3d981122024e02ab51cc8855724525'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:21.930969 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-c5038a24-3bf3-4154-ab3c-711adfb8ca64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b3d981122024e02ab51cc8855724525'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:21.945118 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-c5038a24-3bf3-4154-ab3c-711adfb8ca64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b3d981122024e02ab51cc8855724525'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:22.006463 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1350/2665] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:51:21 2022] DELETE /identity/v3/users/4b3d981122024e02ab51cc8855724525 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:51:22.010993 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a1db1a-d630-4392-8508-d750e91a076c None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:22.011503 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a1db1a-d630-4392-8508-d750e91a076c None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:22.011599 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a1db1a-d630-4392-8508-d750e91a076c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:22.065372 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-84a1db1a-d630-4392-8508-d750e91a076c None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:51:22.066557 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-84a1db1a-d630-4392-8508-d750e91a076c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:51:22.123875 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1316/2666] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:51:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:51:22.133446 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5038a24-3bf3-4154-ab3c-711adfb8ca64 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:22.134842 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-c5038a24-3bf3-4154-ab3c-711adfb8ca64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:51:22.171075 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:51:22.227484 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5038a24-3bf3-4154-ab3c-711adfb8ca64 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:22.231632 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9e51b2-6e84-4655-879c-e58f4c5e9502 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:22.232359 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9e51b2-6e84-4655-879c-e58f4c5e9502 service neutron] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:22.232779 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9e51b2-6e84-4655-879c-e58f4c5e9502 service neutron] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:22.233062 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9e51b2-6e84-4655-879c-e58f4c5e9502 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:22.242363 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9e51b2-6e84-4655-879c-e58f4c5e9502 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:22.243819 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9e51b2-6e84-4655-879c-e58f4c5e9502 service neutron] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:22.253657 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1351/2667] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:51:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:51:22.641544 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a1db1a-d630-4392-8508-d750e91a076c None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:22.642952 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-84a1db1a-d630-4392-8508-d750e91a076c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:51:22.659379 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a1db1a-d630-4392-8508-d750e91a076c None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:22.665260 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ec64d7-f07a-449c-961e-9c79d92795f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:22.666487 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ec64d7-f07a-449c-961e-9c79d92795f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:22.667342 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ec64d7-f07a-449c-961e-9c79d92795f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:22.667620 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ec64d7-f07a-449c-961e-9c79d92795f7 admin admin] PATH_INFO: `/v3/projects/27f784d297fb44d8a214cc1c2f3a95fa` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:22.668961 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ec64d7-f07a-449c-961e-9c79d92795f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27f784d297fb44d8a214cc1c2f3a95fa)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:22.670848 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ec64d7-f07a-449c-961e-9c79d92795f7 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:22.686902 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:51:22.777458 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-b9ec64d7-f07a-449c-961e-9c79d92795f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27f784d297fb44d8a214cc1c2f3a95fa', 'request_id': 'req-b9ec64d7-f07a-449c-961e-9c79d92795f7'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:22.874570 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-b9ec64d7-f07a-449c-961e-9c79d92795f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27f784d297fb44d8a214cc1c2f3a95fa', 'request_id': 'req-b9ec64d7-f07a-449c-961e-9c79d92795f7'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:22.880345 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-b9ec64d7-f07a-449c-961e-9c79d92795f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27f784d297fb44d8a214cc1c2f3a95fa', 'request_id': 'req-b9ec64d7-f07a-449c-961e-9c79d92795f7'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:22.891521 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-b9ec64d7-f07a-449c-961e-9c79d92795f7 admin admin] The token cache is being invalidate because project 27f784d297fb44d8a214cc1c2f3a95fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:51:22.891838 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-b9ec64d7-f07a-449c-961e-9c79d92795f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:22.892506 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-b9ec64d7-f07a-449c-961e-9c79d92795f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:22.894080 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1317/2668] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:51:22 2022] DELETE /identity/v3/projects/27f784d297fb44d8a214cc1c2f3a95fa => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:51:22.900031 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9e51b2-6e84-4655-879c-e58f4c5e9502 service neutron] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:22.912083 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9e51b2-6e84-4655-879c-e58f4c5e9502 service neutron] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:22.968653 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63134d88-ace9-4250-a628-1af8519fd4d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:22.969533 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63134d88-ace9-4250-a628-1af8519fd4d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:22.969814 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63134d88-ace9-4250-a628-1af8519fd4d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:22.970085 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63134d88-ace9-4250-a628-1af8519fd4d6 admin admin] PATH_INFO: `/v3/users/5867aa461249410a8451a7502f3b77ad` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:22.971274 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63134d88-ace9-4250-a628-1af8519fd4d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5867aa461249410a8451a7502f3b77ad)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:22.973236 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63134d88-ace9-4250-a628-1af8519fd4d6 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:23.129096 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-63134d88-ace9-4250-a628-1af8519fd4d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5867aa461249410a8451a7502f3b77ad'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:23.129750 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-63134d88-ace9-4250-a628-1af8519fd4d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5867aa461249410a8451a7502f3b77ad'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:23.130355 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-63134d88-ace9-4250-a628-1af8519fd4d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5867aa461249410a8451a7502f3b77ad'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:23.137699 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-63134d88-ace9-4250-a628-1af8519fd4d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5867aa461249410a8451a7502f3b77ad'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:23.149701 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-63134d88-ace9-4250-a628-1af8519fd4d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5867aa461249410a8451a7502f3b77ad'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:23.269558 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1352/2669] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:51:22 2022] DELETE /identity/v3/users/5867aa461249410a8451a7502f3b77ad => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:51:23.657420 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ec64d7-f07a-449c-961e-9c79d92795f7 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:23.658895 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-b9ec64d7-f07a-449c-961e-9c79d92795f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:51:23.671978 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ec64d7-f07a-449c-961e-9c79d92795f7 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:23.730079 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49db983e-6be1-4072-b35c-2cc379161dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:23.730944 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49db983e-6be1-4072-b35c-2cc379161dd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:23.731232 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49db983e-6be1-4072-b35c-2cc379161dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:23.731509 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49db983e-6be1-4072-b35c-2cc379161dd0 admin admin] PATH_INFO: `/v3/projects/af31ae023fa940d9a65d9f9727cbe832` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:23.732747 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49db983e-6be1-4072-b35c-2cc379161dd0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af31ae023fa940d9a65d9f9727cbe832)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:23.734674 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49db983e-6be1-4072-b35c-2cc379161dd0 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:23.822299 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-49db983e-6be1-4072-b35c-2cc379161dd0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af31ae023fa940d9a65d9f9727cbe832', 'request_id': 'req-49db983e-6be1-4072-b35c-2cc379161dd0'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:23.886002 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-49db983e-6be1-4072-b35c-2cc379161dd0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af31ae023fa940d9a65d9f9727cbe832', 'request_id': 'req-49db983e-6be1-4072-b35c-2cc379161dd0'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:23.891718 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-49db983e-6be1-4072-b35c-2cc379161dd0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af31ae023fa940d9a65d9f9727cbe832', 'request_id': 'req-49db983e-6be1-4072-b35c-2cc379161dd0'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:23.903893 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-49db983e-6be1-4072-b35c-2cc379161dd0 admin admin] The token cache is being invalidate because project af31ae023fa940d9a65d9f9727cbe832 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:51:23.904213 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-49db983e-6be1-4072-b35c-2cc379161dd0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:23.904887 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-49db983e-6be1-4072-b35c-2cc379161dd0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:23.906568 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1318/2670] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:51:23 2022] DELETE /identity/v3/projects/af31ae023fa940d9a65d9f9727cbe832 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:51:23.910526 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63134d88-ace9-4250-a628-1af8519fd4d6 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:23.911994 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-63134d88-ace9-4250-a628-1af8519fd4d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:51:23.924728 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63134d88-ace9-4250-a628-1af8519fd4d6 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:23.941134 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:51:23.982654 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9a873c-5e22-4785-8ac5-6e0d8f908be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:23.983512 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9a873c-5e22-4785-8ac5-6e0d8f908be0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:23.983792 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9a873c-5e22-4785-8ac5-6e0d8f908be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:23.984067 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9a873c-5e22-4785-8ac5-6e0d8f908be0 admin admin] PATH_INFO: `/v3/users/b803ec3c6a994b738c036223ac89adc3` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:23.985406 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9a873c-5e22-4785-8ac5-6e0d8f908be0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b803ec3c6a994b738c036223ac89adc3)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:23.988145 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9a873c-5e22-4785-8ac5-6e0d8f908be0 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:24.168020 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-dd9a873c-5e22-4785-8ac5-6e0d8f908be0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b803ec3c6a994b738c036223ac89adc3'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:24.168601 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-dd9a873c-5e22-4785-8ac5-6e0d8f908be0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b803ec3c6a994b738c036223ac89adc3'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:24.169195 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-dd9a873c-5e22-4785-8ac5-6e0d8f908be0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b803ec3c6a994b738c036223ac89adc3'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:24.175941 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-dd9a873c-5e22-4785-8ac5-6e0d8f908be0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b803ec3c6a994b738c036223ac89adc3'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:24.188269 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-dd9a873c-5e22-4785-8ac5-6e0d8f908be0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b803ec3c6a994b738c036223ac89adc3'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:24.247112 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1353/2671] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:51:23 2022] DELETE /identity/v3/users/b803ec3c6a994b738c036223ac89adc3 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:51:24.671159 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49db983e-6be1-4072-b35c-2cc379161dd0 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:24.672658 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-49db983e-6be1-4072-b35c-2cc379161dd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:51:24.686082 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49db983e-6be1-4072-b35c-2cc379161dd0 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:24.716180 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:51:24.742484 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bf14ec-3354-432a-840d-41ac0f83e60f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:24.743351 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bf14ec-3354-432a-840d-41ac0f83e60f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:24.743640 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bf14ec-3354-432a-840d-41ac0f83e60f admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:24.743922 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bf14ec-3354-432a-840d-41ac0f83e60f admin admin] PATH_INFO: `/v3/projects/f883e059c87c4f7da6e8f5309e6a1b96` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:24.745672 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bf14ec-3354-432a-840d-41ac0f83e60f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f883e059c87c4f7da6e8f5309e6a1b96)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:24.747572 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bf14ec-3354-432a-840d-41ac0f83e60f admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:24.856072 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-c7bf14ec-3354-432a-840d-41ac0f83e60f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f883e059c87c4f7da6e8f5309e6a1b96', 'request_id': 'req-c7bf14ec-3354-432a-840d-41ac0f83e60f'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:24.954898 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-c7bf14ec-3354-432a-840d-41ac0f83e60f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f883e059c87c4f7da6e8f5309e6a1b96', 'request_id': 'req-c7bf14ec-3354-432a-840d-41ac0f83e60f'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:24.961165 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-c7bf14ec-3354-432a-840d-41ac0f83e60f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f883e059c87c4f7da6e8f5309e6a1b96', 'request_id': 'req-c7bf14ec-3354-432a-840d-41ac0f83e60f'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:24.972928 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-c7bf14ec-3354-432a-840d-41ac0f83e60f admin admin] The token cache is being invalidate because project f883e059c87c4f7da6e8f5309e6a1b96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:51:24.973247 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-c7bf14ec-3354-432a-840d-41ac0f83e60f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:24.973887 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-c7bf14ec-3354-432a-840d-41ac0f83e60f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:24.975423 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1319/2672] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:51:24 2022] DELETE /identity/v3/projects/f883e059c87c4f7da6e8f5309e6a1b96 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:51:24.981021 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60501222-c594-4260-a651-c9aebf0328fd None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:24.981496 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60501222-c594-4260-a651-c9aebf0328fd None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:24.981707 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60501222-c594-4260-a651-c9aebf0328fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:25.018718 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-60501222-c594-4260-a651-c9aebf0328fd None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:51:25.019923 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-60501222-c594-4260-a651-c9aebf0328fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:51:25.080450 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1354/2673] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:51:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:51:25.086254 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c1d2d0-3062-44b1-a048-5c9f92cca25e None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:25.086551 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c1d2d0-3062-44b1-a048-5c9f92cca25e None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:25.086838 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c1d2d0-3062-44b1-a048-5c9f92cca25e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:25.106978 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-51c1d2d0-3062-44b1-a048-5c9f92cca25e None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:51:25.108179 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-51c1d2d0-3062-44b1-a048-5c9f92cca25e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:51:25.116078 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1320/2674] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:51:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:51:25.120346 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60501222-c594-4260-a651-c9aebf0328fd None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:25.130021 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60501222-c594-4260-a651-c9aebf0328fd None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:25.134656 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fd6277-08b7-4b9b-85bb-8b6b8bc6b795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:25.135312 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fd6277-08b7-4b9b-85bb-8b6b8bc6b795 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:25.135601 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fd6277-08b7-4b9b-85bb-8b6b8bc6b795 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:25.135882 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fd6277-08b7-4b9b-85bb-8b6b8bc6b795 admin admin] PATH_INFO: `/v3/domains` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:25.136647 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fd6277-08b7-4b9b-85bb-8b6b8bc6b795 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:51:25.136958 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fd6277-08b7-4b9b-85bb-8b6b8bc6b795 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:25.138586 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fd6277-08b7-4b9b-85bb-8b6b8bc6b795 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:25.150533 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1355/2675] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jun 7 14:51:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:51:25.155321 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c1d2d0-3062-44b1-a048-5c9f92cca25e None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:25.164589 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c1d2d0-3062-44b1-a048-5c9f92cca25e None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:25.169952 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec52af73-6b0f-4fa0-b8f4-1563decc0cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:25.170610 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec52af73-6b0f-4fa0-b8f4-1563decc0cb6 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:25.170893 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec52af73-6b0f-4fa0-b8f4-1563decc0cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:25.171172 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec52af73-6b0f-4fa0-b8f4-1563decc0cb6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:25.171870 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec52af73-6b0f-4fa0-b8f4-1563decc0cb6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:25.173789 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec52af73-6b0f-4fa0-b8f4-1563decc0cb6 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:25.299442 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1321/2676] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:51:25 2022] POST /identity/v3/projects => generated 373 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:51:25.303444 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fd6277-08b7-4b9b-85bb-8b6b8bc6b795 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:25.314460 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fd6277-08b7-4b9b-85bb-8b6b8bc6b795 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:25.322101 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:51:25.373081 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a12186-ea3f-42c5-8c0d-ce5b0f8d7013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:25.373755 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a12186-ea3f-42c5-8c0d-ce5b0f8d7013 admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:25.374041 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a12186-ea3f-42c5-8c0d-ce5b0f8d7013 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:25.374318 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a12186-ea3f-42c5-8c0d-ce5b0f8d7013 admin admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:25.375016 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a12186-ea3f-42c5-8c0d-ce5b0f8d7013 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:25.376946 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a12186-ea3f-42c5-8c0d-ce5b0f8d7013 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:25.498062 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1356/2677] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:51:25 2022] POST /identity/v3/users => generated 287 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:51:25.502120 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec52af73-6b0f-4fa0-b8f4-1563decc0cb6 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:25.512281 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec52af73-6b0f-4fa0-b8f4-1563decc0cb6 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:25.516962 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee48f4f-9b5b-49d3-ad6a-14a17f898bfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:25.517617 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee48f4f-9b5b-49d3-ad6a-14a17f898bfa admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:25.517911 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee48f4f-9b5b-49d3-ad6a-14a17f898bfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:25.518195 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee48f4f-9b5b-49d3-ad6a-14a17f898bfa admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:25.518954 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee48f4f-9b5b-49d3-ad6a-14a17f898bfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:25.520584 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee48f4f-9b5b-49d3-ad6a-14a17f898bfa admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:25.531059 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1322/2678] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:51:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:51:25.535058 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a12186-ea3f-42c5-8c0d-ce5b0f8d7013 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:25.547245 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a12186-ea3f-42c5-8c0d-ce5b0f8d7013 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:25.552141 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a1c435-7d01-4c16-b375-12b34fbcad07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:25.552806 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a1c435-7d01-4c16-b375-12b34fbcad07 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:25.553072 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a1c435-7d01-4c16-b375-12b34fbcad07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:25.553353 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a1c435-7d01-4c16-b375-12b34fbcad07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:25.554081 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a1c435-7d01-4c16-b375-12b34fbcad07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:25.555683 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a1c435-7d01-4c16-b375-12b34fbcad07 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:25.569033 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1357/2679] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:51:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:51:25.575761 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee48f4f-9b5b-49d3-ad6a-14a17f898bfa admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:25.591044 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee48f4f-9b5b-49d3-ad6a-14a17f898bfa admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:25.595740 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e4ddb1-6c01-4f4b-875c-45bce0e3cdc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:25.596670 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e4ddb1-6c01-4f4b-875c-45bce0e3cdc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:25.596990 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e4ddb1-6c01-4f4b-875c-45bce0e3cdc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:25.597246 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e4ddb1-6c01-4f4b-875c-45bce0e3cdc7 admin admin] PATH_INFO: `/v3/projects/62d8887963df4571a72bd49ee300d666/users/5a2102cee69d419987963ff9944665bb/roles/3771341b3be641139e752aab690397a4` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:25.613178 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e4ddb1-6c01-4f4b-875c-45bce0e3cdc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62d8887963df4571a72bd49ee300d666, user_id=5a2102cee69d419987963ff9944665bb, role_id=3771341b3be641139e752aab690397a4)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:25.615826 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e4ddb1-6c01-4f4b-875c-45bce0e3cdc7 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:25.699453 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1323/2680] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:51:25 2022] PUT /identity/v3/projects/62d8887963df4571a72bd49ee300d666/users/5a2102cee69d419987963ff9944665bb/roles/3771341b3be641139e752aab690397a4 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:51:25.710496 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b015d8-5f1f-45f6-ac65-b7cf69031a75 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:25.710814 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b015d8-5f1f-45f6-ac65-b7cf69031a75 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:25.711098 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b015d8-5f1f-45f6-ac65-b7cf69031a75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:25.738345 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-50b015d8-5f1f-45f6-ac65-b7cf69031a75 None None] MFA Rules not processed for user `5a2102cee69d419987963ff9944665bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:51:25.742032 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-50b015d8-5f1f-45f6-ac65-b7cf69031a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:51:25.803746 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1358/2681] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:51:25 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:51:25.814749 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e4ddb1-6c01-4f4b-875c-45bce0e3cdc7 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:25.816486 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-68e4ddb1-6c01-4f4b-875c-45bce0e3cdc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:51:25.878622 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e4ddb1-6c01-4f4b-875c-45bce0e3cdc7 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:25.882655 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc538a2-1609-43f4-8dba-c202139d9a32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d81f597cb44c56a966537089bcce4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:25.883303 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc538a2-1609-43f4-8dba-c202139d9a32 service cinder] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:25.883590 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc538a2-1609-43f4-8dba-c202139d9a32 service cinder] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:25.883872 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc538a2-1609-43f4-8dba-c202139d9a32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:25.892983 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc538a2-1609-43f4-8dba-c202139d9a32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:25.894403 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc538a2-1609-43f4-8dba-c202139d9a32 service cinder] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:25.905819 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1324/2682] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:51:25 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:51:25.931465 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b015d8-5f1f-45f6-ac65-b7cf69031a75 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:26.001460 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b015d8-5f1f-45f6-ac65-b7cf69031a75 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:26.005990 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf5c1b8-a3ed-4b4d-a1e1-8fc083a7662c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:26.006863 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf5c1b8-a3ed-4b4d-a1e1-8fc083a7662c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:26.007145 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf5c1b8-a3ed-4b4d-a1e1-8fc083a7662c admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:26.007420 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf5c1b8-a3ed-4b4d-a1e1-8fc083a7662c admin admin] PATH_INFO: `/v3/users/5a2102cee69d419987963ff9944665bb` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:26.008635 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf5c1b8-a3ed-4b4d-a1e1-8fc083a7662c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a2102cee69d419987963ff9944665bb)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:26.010564 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf5c1b8-a3ed-4b4d-a1e1-8fc083a7662c admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:26.031907 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:51:26.213562 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-4bf5c1b8-a3ed-4b4d-a1e1-8fc083a7662c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a2102cee69d419987963ff9944665bb'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:26.214217 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-4bf5c1b8-a3ed-4b4d-a1e1-8fc083a7662c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a2102cee69d419987963ff9944665bb'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:26.214850 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-4bf5c1b8-a3ed-4b4d-a1e1-8fc083a7662c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a2102cee69d419987963ff9944665bb'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:26.222006 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-4bf5c1b8-a3ed-4b4d-a1e1-8fc083a7662c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a2102cee69d419987963ff9944665bb'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:26.247937 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-4bf5c1b8-a3ed-4b4d-a1e1-8fc083a7662c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a2102cee69d419987963ff9944665bb'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:26.342867 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1359/2683] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:51:25 2022] DELETE /identity/v3/users/5a2102cee69d419987963ff9944665bb => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:51:26.350485 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049697c3-dee8-4c69-8be6-df4860127f05 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:26.350844 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049697c3-dee8-4c69-8be6-df4860127f05 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:26.351159 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049697c3-dee8-4c69-8be6-df4860127f05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:26.356912 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:51:26.368814 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-049697c3-dee8-4c69-8be6-df4860127f05 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:51:26.370532 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-049697c3-dee8-4c69-8be6-df4860127f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:51:26.427948 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1325/2684] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:51:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:51:26.437963 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf5c1b8-a3ed-4b4d-a1e1-8fc083a7662c admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:26.439371 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-4bf5c1b8-a3ed-4b4d-a1e1-8fc083a7662c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:51:26.496809 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf5c1b8-a3ed-4b4d-a1e1-8fc083a7662c admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:26.500332 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55eec84-d48c-4ffe-81dc-bc552f739ea3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:26.501664 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55eec84-d48c-4ffe-81dc-bc552f739ea3 service neutron] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:26.501954 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55eec84-d48c-4ffe-81dc-bc552f739ea3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:26.502234 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55eec84-d48c-4ffe-81dc-bc552f739ea3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:26.511836 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55eec84-d48c-4ffe-81dc-bc552f739ea3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:26.513296 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55eec84-d48c-4ffe-81dc-bc552f739ea3 service neutron] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:26.522737 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1360/2685] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:51:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:51:26.897492 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049697c3-dee8-4c69-8be6-df4860127f05 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:26.899027 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-049697c3-dee8-4c69-8be6-df4860127f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:51:26.912479 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049697c3-dee8-4c69-8be6-df4860127f05 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:26.917808 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e041fb62-73a2-499c-9b13-4aca5fc2ea0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:26.918675 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e041fb62-73a2-499c-9b13-4aca5fc2ea0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:26.918964 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e041fb62-73a2-499c-9b13-4aca5fc2ea0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:26.919241 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e041fb62-73a2-499c-9b13-4aca5fc2ea0c admin admin] PATH_INFO: `/v3/projects/62d8887963df4571a72bd49ee300d666` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:26.920622 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e041fb62-73a2-499c-9b13-4aca5fc2ea0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=62d8887963df4571a72bd49ee300d666)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:26.925985 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e041fb62-73a2-499c-9b13-4aca5fc2ea0c admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:27.016799 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-e041fb62-73a2-499c-9b13-4aca5fc2ea0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62d8887963df4571a72bd49ee300d666', 'request_id': 'req-e041fb62-73a2-499c-9b13-4aca5fc2ea0c'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:27.073442 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-e041fb62-73a2-499c-9b13-4aca5fc2ea0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62d8887963df4571a72bd49ee300d666', 'request_id': 'req-e041fb62-73a2-499c-9b13-4aca5fc2ea0c'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:27.079738 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-e041fb62-73a2-499c-9b13-4aca5fc2ea0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62d8887963df4571a72bd49ee300d666', 'request_id': 'req-e041fb62-73a2-499c-9b13-4aca5fc2ea0c'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:27.091366 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-e041fb62-73a2-499c-9b13-4aca5fc2ea0c admin admin] The token cache is being invalidate because project 62d8887963df4571a72bd49ee300d666 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:51:27.091659 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-e041fb62-73a2-499c-9b13-4aca5fc2ea0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:27.092304 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-e041fb62-73a2-499c-9b13-4aca5fc2ea0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:27.093889 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1326/2686] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:51:26 2022] DELETE /identity/v3/projects/62d8887963df4571a72bd49ee300d666 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:51:27.098715 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b98a4-5a2b-4b72-b2db-b92d1adfb88e None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:27.099206 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b98a4-5a2b-4b72-b2db-b92d1adfb88e None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:27.099317 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b98a4-5a2b-4b72-b2db-b92d1adfb88e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:27.117231 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-0f0b98a4-5a2b-4b72-b2db-b92d1adfb88e None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:51:27.118433 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-0f0b98a4-5a2b-4b72-b2db-b92d1adfb88e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:51:27.179424 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1361/2687] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:51:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:51:27.190526 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b533304d-f38e-402a-be27-4e2b63f0d3d4 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:27.190816 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b533304d-f38e-402a-be27-4e2b63f0d3d4 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:27.191128 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b533304d-f38e-402a-be27-4e2b63f0d3d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:27.210772 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-b533304d-f38e-402a-be27-4e2b63f0d3d4 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:51:27.211991 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-b533304d-f38e-402a-be27-4e2b63f0d3d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:51:27.219768 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1327/2688] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:51:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:51:27.223983 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0b98a4-5a2b-4b72-b2db-b92d1adfb88e None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:27.233801 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0b98a4-5a2b-4b72-b2db-b92d1adfb88e None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:27.238476 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca464d5-e0d9-4de0-8f20-47a617f11035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:27.239126 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca464d5-e0d9-4de0-8f20-47a617f11035 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:27.239411 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca464d5-e0d9-4de0-8f20-47a617f11035 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:27.239690 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca464d5-e0d9-4de0-8f20-47a617f11035 admin admin] PATH_INFO: `/v3/domains` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:27.240448 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca464d5-e0d9-4de0-8f20-47a617f11035 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:51:27.240761 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca464d5-e0d9-4de0-8f20-47a617f11035 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:27.242350 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca464d5-e0d9-4de0-8f20-47a617f11035 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:27.255543 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1362/2689] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jun 7 14:51:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:51:27.259621 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b533304d-f38e-402a-be27-4e2b63f0d3d4 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:27.270924 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b533304d-f38e-402a-be27-4e2b63f0d3d4 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:27.274683 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b2f171-5494-4768-82e5-921b3bda994c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:27.275344 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b2f171-5494-4768-82e5-921b3bda994c admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:27.275633 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b2f171-5494-4768-82e5-921b3bda994c admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:27.276820 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b2f171-5494-4768-82e5-921b3bda994c admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:27.277546 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b2f171-5494-4768-82e5-921b3bda994c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:27.282992 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b2f171-5494-4768-82e5-921b3bda994c admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:27.379073 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1328/2690] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:51:27 2022] POST /identity/v3/projects => generated 389 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:51:27.383159 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca464d5-e0d9-4de0-8f20-47a617f11035 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:27.393900 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca464d5-e0d9-4de0-8f20-47a617f11035 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:27.421048 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:51:27.452734 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edebb88-5d38-4522-a377-368ce6da68bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:27.453406 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edebb88-5d38-4522-a377-368ce6da68bd admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:27.453691 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edebb88-5d38-4522-a377-368ce6da68bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:27.453973 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edebb88-5d38-4522-a377-368ce6da68bd admin admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:27.454657 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edebb88-5d38-4522-a377-368ce6da68bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:27.456652 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edebb88-5d38-4522-a377-368ce6da68bd admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:27.532172 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1363/2691] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:51:27 2022] POST /identity/v3/users => generated 295 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:51:27.536260 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b2f171-5494-4768-82e5-921b3bda994c admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:27.547178 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b2f171-5494-4768-82e5-921b3bda994c admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:27.551801 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea45c4b-476e-4665-b166-365da3b0e938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:27.552491 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea45c4b-476e-4665-b166-365da3b0e938 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:27.552779 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea45c4b-476e-4665-b166-365da3b0e938 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:27.553063 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea45c4b-476e-4665-b166-365da3b0e938 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:27.553793 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea45c4b-476e-4665-b166-365da3b0e938 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:27.555362 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea45c4b-476e-4665-b166-365da3b0e938 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:27.566190 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1329/2692] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:51:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 14:51:27.570270 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edebb88-5d38-4522-a377-368ce6da68bd admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:27.579979 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edebb88-5d38-4522-a377-368ce6da68bd admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:27.584545 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f75ff6-4376-40d9-9197-8e78372aafd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:27.585189 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f75ff6-4376-40d9-9197-8e78372aafd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:27.585474 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f75ff6-4376-40d9-9197-8e78372aafd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:27.585748 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f75ff6-4376-40d9-9197-8e78372aafd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:27.586493 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f75ff6-4376-40d9-9197-8e78372aafd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:27.588067 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f75ff6-4376-40d9-9197-8e78372aafd0 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:27.603078 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1364/2693] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:51:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:51:27.607663 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea45c4b-476e-4665-b166-365da3b0e938 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:27.620547 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea45c4b-476e-4665-b166-365da3b0e938 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:27.626338 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12024d8d-9e64-49c6-a8bb-f1abd657a061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:27.627267 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12024d8d-9e64-49c6-a8bb-f1abd657a061 admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:27.627557 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12024d8d-9e64-49c6-a8bb-f1abd657a061 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:27.627841 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12024d8d-9e64-49c6-a8bb-f1abd657a061 admin admin] PATH_INFO: `/v3/projects/e2765a6d16fe46a6960e2949d247eeb5/users/7577d0f527d648c3aa7e768fbea37d7f/roles/3771341b3be641139e752aab690397a4` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:27.642758 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12024d8d-9e64-49c6-a8bb-f1abd657a061 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2765a6d16fe46a6960e2949d247eeb5, user_id=7577d0f527d648c3aa7e768fbea37d7f, role_id=3771341b3be641139e752aab690397a4)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:27.645439 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12024d8d-9e64-49c6-a8bb-f1abd657a061 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:27.737843 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1330/2694] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:51:27 2022] PUT /identity/v3/projects/e2765a6d16fe46a6960e2949d247eeb5/users/7577d0f527d648c3aa7e768fbea37d7f/roles/3771341b3be641139e752aab690397a4 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:51:27.750417 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5841bfb6-7f8d-44fa-89b7-bd4a141509e0 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:27.750821 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5841bfb6-7f8d-44fa-89b7-bd4a141509e0 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:27.751105 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5841bfb6-7f8d-44fa-89b7-bd4a141509e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:27.777621 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-5841bfb6-7f8d-44fa-89b7-bd4a141509e0 None None] MFA Rules not processed for user `7577d0f527d648c3aa7e768fbea37d7f`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:51:27.778836 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-5841bfb6-7f8d-44fa-89b7-bd4a141509e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:51:27.853728 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1365/2695] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:51:27 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:51:27.864784 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12024d8d-9e64-49c6-a8bb-f1abd657a061 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:27.866519 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-12024d8d-9e64-49c6-a8bb-f1abd657a061 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:51:27.894022 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:51:27.923409 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12024d8d-9e64-49c6-a8bb-f1abd657a061 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:27.927171 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fcbb37-0f42-4db5-95af-60fbc28e3617 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d81f597cb44c56a966537089bcce4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:27.927820 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fcbb37-0f42-4db5-95af-60fbc28e3617 service cinder] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:27.928105 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fcbb37-0f42-4db5-95af-60fbc28e3617 service cinder] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:27.928436 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fcbb37-0f42-4db5-95af-60fbc28e3617 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:27.938006 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fcbb37-0f42-4db5-95af-60fbc28e3617 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:27.939440 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fcbb37-0f42-4db5-95af-60fbc28e3617 service cinder] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:27.949674 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1331/2696] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:51:27 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:51:33.903990 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5841bfb6-7f8d-44fa-89b7-bd4a141509e0 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:33.913631 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5841bfb6-7f8d-44fa-89b7-bd4a141509e0 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:33.920177 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2926a0-14c2-46d0-bcf8-ec4a83073d8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d81f597cb44c56a966537089bcce4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:33.921289 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2926a0-14c2-46d0-bcf8-ec4a83073d8e service cinder] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:33.921716 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2926a0-14c2-46d0-bcf8-ec4a83073d8e service cinder] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:33.922139 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2926a0-14c2-46d0-bcf8-ec4a83073d8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:33.925335 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-bf2926a0-14c2-46d0-bcf8-ec4a83073d8e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:51:34.013298 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2926a0-14c2-46d0-bcf8-ec4a83073d8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:34.014736 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2926a0-14c2-46d0-bcf8-ec4a83073d8e service cinder] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:34.037261 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1366/2697] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:51:33 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:51:38.005605 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=16 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:51:41.775641 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fcbb37-0f42-4db5-95af-60fbc28e3617 service cinder] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:41.778160 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-09fcbb37-0f42-4db5-95af-60fbc28e3617 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:51:41.862304 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fcbb37-0f42-4db5-95af-60fbc28e3617 service cinder] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:41.868448 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aa7f68-007c-4272-8088-f932943eae8e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e857db4902e42989d0d040bbc8517c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:41.869206 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aa7f68-007c-4272-8088-f932943eae8e service nova] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:41.869574 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aa7f68-007c-4272-8088-f932943eae8e service nova] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:41.869927 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aa7f68-007c-4272-8088-f932943eae8e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:41.871922 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-c4aa7f68-007c-4272-8088-f932943eae8e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:51:41.886019 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aa7f68-007c-4272-8088-f932943eae8e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:41.887743 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aa7f68-007c-4272-8088-f932943eae8e service nova] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:41.900075 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1332/2698] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:51:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:51:44.035811 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=94,keystone:INSERT=3 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:51:51.894712 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=34 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:51:54.137069 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2926a0-14c2-46d0-bcf8-ec4a83073d8e service cinder] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:54.138922 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-bf2926a0-14c2-46d0-bcf8-ec4a83073d8e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:51:54.196776 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2926a0-14c2-46d0-bcf8-ec4a83073d8e service cinder] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:54.200465 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36270826-baa1-4c63-b1f4-496f8dfc58f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:54.201113 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36270826-baa1-4c63-b1f4-496f8dfc58f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:54.201398 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36270826-baa1-4c63-b1f4-496f8dfc58f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:54.201676 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36270826-baa1-4c63-b1f4-496f8dfc58f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:54.203539 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-36270826-baa1-4c63-b1f4-496f8dfc58f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:51:54.264943 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36270826-baa1-4c63-b1f4-496f8dfc58f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:54.266384 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36270826-baa1-4c63-b1f4-496f8dfc58f6 service neutron] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:54.276229 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1367/2699] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:51:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:51:54.610880 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aa7f68-007c-4272-8088-f932943eae8e service nova] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:54.612480 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-c4aa7f68-007c-4272-8088-f932943eae8e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:51:54.632206 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aa7f68-007c-4272-8088-f932943eae8e service nova] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:54.637073 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84673e5d-11fa-4dd4-b3b9-63da93ac7cdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:54.637961 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84673e5d-11fa-4dd4-b3b9-63da93ac7cdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:54.638316 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84673e5d-11fa-4dd4-b3b9-63da93ac7cdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:54.638597 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84673e5d-11fa-4dd4-b3b9-63da93ac7cdb admin admin] PATH_INFO: `/v3/users/e1c23b5f03f6445e9a10b93fb775c60c` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:54.639858 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84673e5d-11fa-4dd4-b3b9-63da93ac7cdb admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1c23b5f03f6445e9a10b93fb775c60c)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:54.641854 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84673e5d-11fa-4dd4-b3b9-63da93ac7cdb admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:54.855171 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-84673e5d-11fa-4dd4-b3b9-63da93ac7cdb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1c23b5f03f6445e9a10b93fb775c60c'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:54.855903 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-84673e5d-11fa-4dd4-b3b9-63da93ac7cdb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1c23b5f03f6445e9a10b93fb775c60c'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:54.856548 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-84673e5d-11fa-4dd4-b3b9-63da93ac7cdb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1c23b5f03f6445e9a10b93fb775c60c'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:54.864976 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-84673e5d-11fa-4dd4-b3b9-63da93ac7cdb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1c23b5f03f6445e9a10b93fb775c60c'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:54.877657 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-84673e5d-11fa-4dd4-b3b9-63da93ac7cdb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1c23b5f03f6445e9a10b93fb775c60c'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:54.943316 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1333/2700] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:51:54 2022] DELETE /identity/v3/users/e1c23b5f03f6445e9a10b93fb775c60c => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:51:54.948224 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ec2d94-fa3c-415c-bc38-48b84c961b6d None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:54.948554 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ec2d94-fa3c-415c-bc38-48b84c961b6d None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:54.948832 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ec2d94-fa3c-415c-bc38-48b84c961b6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:54.968333 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-21ec2d94-fa3c-415c-bc38-48b84c961b6d None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:51:54.969649 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-21ec2d94-fa3c-415c-bc38-48b84c961b6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:51:55.031322 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1368/2701] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:51:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:51:55.041372 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84673e5d-11fa-4dd4-b3b9-63da93ac7cdb admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:55.042800 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-84673e5d-11fa-4dd4-b3b9-63da93ac7cdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:51:55.101876 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84673e5d-11fa-4dd4-b3b9-63da93ac7cdb admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:55.105536 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dca3220-1452-4fb2-8551-f5248f8e2023 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:55.106128 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dca3220-1452-4fb2-8551-f5248f8e2023 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:55.106409 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dca3220-1452-4fb2-8551-f5248f8e2023 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:55.106684 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dca3220-1452-4fb2-8551-f5248f8e2023 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:55.115937 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dca3220-1452-4fb2-8551-f5248f8e2023 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:55.119157 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dca3220-1452-4fb2-8551-f5248f8e2023 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:55.130402 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1334/2702] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:51:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:51:55.601735 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ec2d94-fa3c-415c-bc38-48b84c961b6d None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:55.603289 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-21ec2d94-fa3c-415c-bc38-48b84c961b6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:51:55.616510 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ec2d94-fa3c-415c-bc38-48b84c961b6d None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:55.637742 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec8e83c-a03c-48ea-ae8b-d58bc7e059c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:55.637742 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec8e83c-a03c-48ea-ae8b-d58bc7e059c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:55.637742 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec8e83c-a03c-48ea-ae8b-d58bc7e059c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:55.637742 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec8e83c-a03c-48ea-ae8b-d58bc7e059c6 admin admin] PATH_INFO: `/v3/projects/8c1eee16d77d46528ba44f3cad4c5095` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:55.638700 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec8e83c-a03c-48ea-ae8b-d58bc7e059c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c1eee16d77d46528ba44f3cad4c5095)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:55.649054 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec8e83c-a03c-48ea-ae8b-d58bc7e059c6 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:55.774048 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:51:55.777399 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-9ec8e83c-a03c-48ea-ae8b-d58bc7e059c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c1eee16d77d46528ba44f3cad4c5095', 'request_id': 'req-9ec8e83c-a03c-48ea-ae8b-d58bc7e059c6'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:55.840114 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-9ec8e83c-a03c-48ea-ae8b-d58bc7e059c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c1eee16d77d46528ba44f3cad4c5095', 'request_id': 'req-9ec8e83c-a03c-48ea-ae8b-d58bc7e059c6'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:55.847045 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-9ec8e83c-a03c-48ea-ae8b-d58bc7e059c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c1eee16d77d46528ba44f3cad4c5095', 'request_id': 'req-9ec8e83c-a03c-48ea-ae8b-d58bc7e059c6'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:55.859289 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-9ec8e83c-a03c-48ea-ae8b-d58bc7e059c6 admin admin] The token cache is being invalidate because project 8c1eee16d77d46528ba44f3cad4c5095 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:51:55.859581 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-9ec8e83c-a03c-48ea-ae8b-d58bc7e059c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:55.860285 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-9ec8e83c-a03c-48ea-ae8b-d58bc7e059c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:55.862256 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1369/2703] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:51:55 2022] DELETE /identity/v3/projects/8c1eee16d77d46528ba44f3cad4c5095 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:51:55.869383 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bd3c7c-a238-4a15-af9f-df46fab9d527 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:55.869383 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bd3c7c-a238-4a15-af9f-df46fab9d527 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:55.869383 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bd3c7c-a238-4a15-af9f-df46fab9d527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:55.897584 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-01bd3c7c-a238-4a15-af9f-df46fab9d527 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:51:55.904089 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-01bd3c7c-a238-4a15-af9f-df46fab9d527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:51:55.962790 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1335/2704] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:51:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:51:55.971277 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee7f1ef-472b-49aa-9d90-3067f22155a0 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:55.971689 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee7f1ef-472b-49aa-9d90-3067f22155a0 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:55.971981 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee7f1ef-472b-49aa-9d90-3067f22155a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:55.999934 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-1ee7f1ef-472b-49aa-9d90-3067f22155a0 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:51:56.001233 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-1ee7f1ef-472b-49aa-9d90-3067f22155a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:51:56.012208 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1370/2705] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:51:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:51:56.014547 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bd3c7c-a238-4a15-af9f-df46fab9d527 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:56.025079 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bd3c7c-a238-4a15-af9f-df46fab9d527 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:56.031687 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b475fd43-b268-4d1c-b911-db9e9783dc85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:56.032435 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b475fd43-b268-4d1c-b911-db9e9783dc85 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:56.032697 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b475fd43-b268-4d1c-b911-db9e9783dc85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:56.033169 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b475fd43-b268-4d1c-b911-db9e9783dc85 admin admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:56.033986 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b475fd43-b268-4d1c-b911-db9e9783dc85 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:51:56.034473 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b475fd43-b268-4d1c-b911-db9e9783dc85 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:56.036305 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b475fd43-b268-4d1c-b911-db9e9783dc85 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:56.040301 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:51:56.051961 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1336/2706] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jun 7 14:51:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:51:56.056353 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee7f1ef-472b-49aa-9d90-3067f22155a0 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:56.074210 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee7f1ef-472b-49aa-9d90-3067f22155a0 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:56.086857 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc2c1b0-e5e3-416f-8e0d-ef4c309ef49e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:56.088116 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc2c1b0-e5e3-416f-8e0d-ef4c309ef49e admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:56.098023 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc2c1b0-e5e3-416f-8e0d-ef4c309ef49e admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:56.098023 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc2c1b0-e5e3-416f-8e0d-ef4c309ef49e admin admin] PATH_INFO: `/v3/projects` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:56.098023 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc2c1b0-e5e3-416f-8e0d-ef4c309ef49e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:56.098023 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc2c1b0-e5e3-416f-8e0d-ef4c309ef49e admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:56.193033 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1371/2707] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:51:56 2022] POST /identity/v3/projects => generated 363 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:51:56.194287 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b475fd43-b268-4d1c-b911-db9e9783dc85 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:56.211405 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b475fd43-b268-4d1c-b911-db9e9783dc85 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:56.285662 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5f36f8-9056-45de-87d1-07544d5ca083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:56.286347 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f36f8-9056-45de-87d1-07544d5ca083 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:56.286634 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f36f8-9056-45de-87d1-07544d5ca083 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:56.286915 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f36f8-9056-45de-87d1-07544d5ca083 admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:56.287627 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5f36f8-9056-45de-87d1-07544d5ca083 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:56.289618 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5f36f8-9056-45de-87d1-07544d5ca083 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:56.382664 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1337/2708] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:51:56 2022] POST /identity/v3/users => generated 282 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:51:56.383928 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc2c1b0-e5e3-416f-8e0d-ef4c309ef49e admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:56.402170 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc2c1b0-e5e3-416f-8e0d-ef4c309ef49e admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:56.407881 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab06c034-9c40-4d86-af9a-ddc1841d9ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:56.409132 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab06c034-9c40-4d86-af9a-ddc1841d9ad7 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:56.409132 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab06c034-9c40-4d86-af9a-ddc1841d9ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:56.409465 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab06c034-9c40-4d86-af9a-ddc1841d9ad7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:56.410047 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab06c034-9c40-4d86-af9a-ddc1841d9ad7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:56.411952 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab06c034-9c40-4d86-af9a-ddc1841d9ad7 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:56.426325 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1372/2709] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:51:56 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:51:56.430423 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5f36f8-9056-45de-87d1-07544d5ca083 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:56.442052 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5f36f8-9056-45de-87d1-07544d5ca083 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:56.448150 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05af169-407c-4de8-ac97-e7ded97cc6e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:56.448950 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05af169-407c-4de8-ac97-e7ded97cc6e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:56.449309 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05af169-407c-4de8-ac97-e7ded97cc6e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:56.449724 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05af169-407c-4de8-ac97-e7ded97cc6e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:56.450540 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05af169-407c-4de8-ac97-e7ded97cc6e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:56.452288 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05af169-407c-4de8-ac97-e7ded97cc6e9 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:56.467155 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1338/2710] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:51:56 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:51:56.483998 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab06c034-9c40-4d86-af9a-ddc1841d9ad7 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:56.494954 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab06c034-9c40-4d86-af9a-ddc1841d9ad7 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:56.501892 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cc554a-590c-41b4-bbbd-9760a22cf0cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:56.502469 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cc554a-590c-41b4-bbbd-9760a22cf0cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:56.502838 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cc554a-590c-41b4-bbbd-9760a22cf0cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:56.503191 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cc554a-590c-41b4-bbbd-9760a22cf0cf admin admin] PATH_INFO: `/v3/projects/2581b55726844ce1b36f318ca0e8259d/users/f46eae035e894a09a6eaced29cc5c43a/roles/3771341b3be641139e752aab690397a4` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:56.519994 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cc554a-590c-41b4-bbbd-9760a22cf0cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=2581b55726844ce1b36f318ca0e8259d, user_id=f46eae035e894a09a6eaced29cc5c43a, role_id=3771341b3be641139e752aab690397a4)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:56.522815 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cc554a-590c-41b4-bbbd-9760a22cf0cf admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:56.634416 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1373/2711] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:51:56 2022] PUT /identity/v3/projects/2581b55726844ce1b36f318ca0e8259d/users/f46eae035e894a09a6eaced29cc5c43a/roles/3771341b3be641139e752aab690397a4 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:51:56.638809 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1d813a-c653-4d71-a3db-2a5fb2b3e3fe None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:56.640312 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1d813a-c653-4d71-a3db-2a5fb2b3e3fe None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:56.640652 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1d813a-c653-4d71-a3db-2a5fb2b3e3fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:56.671818 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-3b1d813a-c653-4d71-a3db-2a5fb2b3e3fe None None] MFA Rules not processed for user `f46eae035e894a09a6eaced29cc5c43a`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:51:56.673410 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-3b1d813a-c653-4d71-a3db-2a5fb2b3e3fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:51:56.748873 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1339/2712] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:51:56 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:51:56.760209 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cc554a-590c-41b4-bbbd-9760a22cf0cf admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:56.761986 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-49cc554a-590c-41b4-bbbd-9760a22cf0cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:51:56.851248 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cc554a-590c-41b4-bbbd-9760a22cf0cf admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:56.855274 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b0a9d0-66b4-4ab1-bae0-1304a77f2e26 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d81f597cb44c56a966537089bcce4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:56.855937 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b0a9d0-66b4-4ab1-bae0-1304a77f2e26 service cinder] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:56.856224 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b0a9d0-66b4-4ab1-bae0-1304a77f2e26 service cinder] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:56.864456 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b0a9d0-66b4-4ab1-bae0-1304a77f2e26 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:56.865807 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b0a9d0-66b4-4ab1-bae0-1304a77f2e26 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:56.867277 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b0a9d0-66b4-4ab1-bae0-1304a77f2e26 service cinder] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:56.877961 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1374/2713] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:51:56 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:51:57.989279 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1d813a-c653-4d71-a3db-2a5fb2b3e3fe None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:57.989279 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-3b1d813a-c653-4d71-a3db-2a5fb2b3e3fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:51:58.073543 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1d813a-c653-4d71-a3db-2a5fb2b3e3fe None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:58.082490 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2f597f-dba9-44a3-bd14-36aece2a82fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:58.083238 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2f597f-dba9-44a3-bd14-36aece2a82fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:58.083532 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2f597f-dba9-44a3-bd14-36aece2a82fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:58.083865 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2f597f-dba9-44a3-bd14-36aece2a82fb admin admin] PATH_INFO: `/v3/users/7577d0f527d648c3aa7e768fbea37d7f` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:58.085376 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2f597f-dba9-44a3-bd14-36aece2a82fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=7577d0f527d648c3aa7e768fbea37d7f)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:58.087478 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2f597f-dba9-44a3-bd14-36aece2a82fb admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:58.090305 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:51:58.202101 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b0a9d0-66b4-4ab1-bae0-1304a77f2e26 service cinder] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:58.204823 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-79b0a9d0-66b4-4ab1-bae0-1304a77f2e26 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:51:58.226716 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b0a9d0-66b4-4ab1-bae0-1304a77f2e26 service cinder] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:58.231254 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913e2448-289a-4b7d-bde7-4ebf7e7ea93b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:58.232125 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913e2448-289a-4b7d-bde7-4ebf7e7ea93b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:58.232434 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913e2448-289a-4b7d-bde7-4ebf7e7ea93b admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:58.232715 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913e2448-289a-4b7d-bde7-4ebf7e7ea93b admin admin] PATH_INFO: `/v3/users/ed3353d3d8a242e1a6b54d31adf97a6d` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:58.233901 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913e2448-289a-4b7d-bde7-4ebf7e7ea93b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed3353d3d8a242e1a6b54d31adf97a6d)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:58.235791 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913e2448-289a-4b7d-bde7-4ebf7e7ea93b admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:58.305712 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-bb2f597f-dba9-44a3-bd14-36aece2a82fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7577d0f527d648c3aa7e768fbea37d7f'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:58.306627 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-bb2f597f-dba9-44a3-bd14-36aece2a82fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7577d0f527d648c3aa7e768fbea37d7f'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:58.307305 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-bb2f597f-dba9-44a3-bd14-36aece2a82fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7577d0f527d648c3aa7e768fbea37d7f'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:58.317784 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-bb2f597f-dba9-44a3-bd14-36aece2a82fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7577d0f527d648c3aa7e768fbea37d7f'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:58.332663 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-bb2f597f-dba9-44a3-bd14-36aece2a82fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7577d0f527d648c3aa7e768fbea37d7f'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:58.430405 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-913e2448-289a-4b7d-bde7-4ebf7e7ea93b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed3353d3d8a242e1a6b54d31adf97a6d'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:58.431047 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-913e2448-289a-4b7d-bde7-4ebf7e7ea93b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed3353d3d8a242e1a6b54d31adf97a6d'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:58.431652 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-913e2448-289a-4b7d-bde7-4ebf7e7ea93b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed3353d3d8a242e1a6b54d31adf97a6d'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:58.433272 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:DELETE=11,keystone:SELECT=86,keystone:INSERT=3 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:51:58.440681 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1340/2714] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:51:57 2022] DELETE /identity/v3/users/7577d0f527d648c3aa7e768fbea37d7f => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:51:58.444527 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-913e2448-289a-4b7d-bde7-4ebf7e7ea93b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed3353d3d8a242e1a6b54d31adf97a6d'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:58.445265 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56a7ee9-6d0f-479c-9eae-5a3e177e1304 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:58.445597 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56a7ee9-6d0f-479c-9eae-5a3e177e1304 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:58.445885 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56a7ee9-6d0f-479c-9eae-5a3e177e1304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:58.458247 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-913e2448-289a-4b7d-bde7-4ebf7e7ea93b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed3353d3d8a242e1a6b54d31adf97a6d'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:58.463359 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-b56a7ee9-6d0f-479c-9eae-5a3e177e1304 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:51:58.464600 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-b56a7ee9-6d0f-479c-9eae-5a3e177e1304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:51:58.512533 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1375/2715] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:51:58 2022] DELETE /identity/v3/users/ed3353d3d8a242e1a6b54d31adf97a6d => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:51:58.525040 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1341/2716] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:51:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:51:58.535092 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913e2448-289a-4b7d-bde7-4ebf7e7ea93b admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:58.536659 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-913e2448-289a-4b7d-bde7-4ebf7e7ea93b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:51:58.596095 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913e2448-289a-4b7d-bde7-4ebf7e7ea93b admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:58.599985 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ddef59-1b79-4160-8ff4-a66acbdfc89f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:58.600694 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ddef59-1b79-4160-8ff4-a66acbdfc89f service neutron] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:58.600984 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ddef59-1b79-4160-8ff4-a66acbdfc89f service neutron] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:58.601268 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ddef59-1b79-4160-8ff4-a66acbdfc89f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:58.667262 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ddef59-1b79-4160-8ff4-a66acbdfc89f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:58.668760 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ddef59-1b79-4160-8ff4-a66acbdfc89f service neutron] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:58.679417 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1376/2717] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:51:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:51:58.862605 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56a7ee9-6d0f-479c-9eae-5a3e177e1304 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:58.864059 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-b56a7ee9-6d0f-479c-9eae-5a3e177e1304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:51:58.882827 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56a7ee9-6d0f-479c-9eae-5a3e177e1304 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:58.888508 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8048893-fd94-4ae7-bb84-0165ef287fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:58.890474 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8048893-fd94-4ae7-bb84-0165ef287fc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:58.890830 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8048893-fd94-4ae7-bb84-0165ef287fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:58.891186 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8048893-fd94-4ae7-bb84-0165ef287fc0 admin admin] PATH_INFO: `/v3/projects/6714e8cfaee34ac3af916af5e4bb1031` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:58.892602 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8048893-fd94-4ae7-bb84-0165ef287fc0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6714e8cfaee34ac3af916af5e4bb1031)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:58.895114 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8048893-fd94-4ae7-bb84-0165ef287fc0 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:59.034695 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-f8048893-fd94-4ae7-bb84-0165ef287fc0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6714e8cfaee34ac3af916af5e4bb1031', 'request_id': 'req-f8048893-fd94-4ae7-bb84-0165ef287fc0'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:59.114239 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-f8048893-fd94-4ae7-bb84-0165ef287fc0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6714e8cfaee34ac3af916af5e4bb1031', 'request_id': 'req-f8048893-fd94-4ae7-bb84-0165ef287fc0'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:59.120919 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-f8048893-fd94-4ae7-bb84-0165ef287fc0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6714e8cfaee34ac3af916af5e4bb1031', 'request_id': 'req-f8048893-fd94-4ae7-bb84-0165ef287fc0'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:59.133185 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-f8048893-fd94-4ae7-bb84-0165ef287fc0 admin admin] The token cache is being invalidate because project 6714e8cfaee34ac3af916af5e4bb1031 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:51:59.133477 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-f8048893-fd94-4ae7-bb84-0165ef287fc0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:59.134151 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-f8048893-fd94-4ae7-bb84-0165ef287fc0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:59.135699 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1342/2718] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:51:58 2022] DELETE /identity/v3/projects/6714e8cfaee34ac3af916af5e4bb1031 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:51:59.141601 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cd3609-ddb7-4f3a-8ddf-4f4ccb5952e2 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:59.141893 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cd3609-ddb7-4f3a-8ddf-4f4ccb5952e2 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:59.142172 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cd3609-ddb7-4f3a-8ddf-4f4ccb5952e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:59.163114 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-26cd3609-ddb7-4f3a-8ddf-4f4ccb5952e2 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:51:59.163363 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8048893-fd94-4ae7-bb84-0165ef287fc0 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:59.164725 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-26cd3609-ddb7-4f3a-8ddf-4f4ccb5952e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:51:59.165018 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-f8048893-fd94-4ae7-bb84-0165ef287fc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:51:59.175909 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:51:59.246833 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8048893-fd94-4ae7-bb84-0165ef287fc0 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:59.247732 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1377/2719] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:51:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:51:59.254015 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a474995-8630-425e-bbd8-b1848733a879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:59.255157 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a474995-8630-425e-bbd8-b1848733a879 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:59.255396 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a474995-8630-425e-bbd8-b1848733a879 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:59.255683 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a474995-8630-425e-bbd8-b1848733a879 admin admin] PATH_INFO: `/v3/projects/e2765a6d16fe46a6960e2949d247eeb5` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:59.256960 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a474995-8630-425e-bbd8-b1848733a879 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2765a6d16fe46a6960e2949d247eeb5)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:59.257559 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc591f8-6602-410c-8d64-8c049f941ecb None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:59.258568 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc591f8-6602-410c-8d64-8c049f941ecb None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:59.258867 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc591f8-6602-410c-8d64-8c049f941ecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:59.259830 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a474995-8630-425e-bbd8-b1848733a879 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:59.280766 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-1bc591f8-6602-410c-8d64-8c049f941ecb None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:51:59.281973 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-1bc591f8-6602-410c-8d64-8c049f941ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:51:59.290023 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1378/2720] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:51:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:51:59.294306 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc591f8-6602-410c-8d64-8c049f941ecb None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:59.303800 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc591f8-6602-410c-8d64-8c049f941ecb None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:59.308613 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe819312-cf26-49bf-bdbb-c6d1b9514e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:59.315096 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe819312-cf26-49bf-bdbb-c6d1b9514e16 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:59.315096 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe819312-cf26-49bf-bdbb-c6d1b9514e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:59.315096 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe819312-cf26-49bf-bdbb-c6d1b9514e16 admin admin] PATH_INFO: `/v3/domains` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:59.315096 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe819312-cf26-49bf-bdbb-c6d1b9514e16 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:51:59.315096 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe819312-cf26-49bf-bdbb-c6d1b9514e16 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:59.315096 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe819312-cf26-49bf-bdbb-c6d1b9514e16 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:59.315661 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:51:59.325172 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1379/2721] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jun 7 14:51:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:51:59.329549 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe819312-cf26-49bf-bdbb-c6d1b9514e16 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:59.339164 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe819312-cf26-49bf-bdbb-c6d1b9514e16 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:59.344210 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26acb6d9-7429-4a6c-86ca-0d70b4402fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:59.344952 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26acb6d9-7429-4a6c-86ca-0d70b4402fb4 admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:59.345241 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26acb6d9-7429-4a6c-86ca-0d70b4402fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:59.345600 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26acb6d9-7429-4a6c-86ca-0d70b4402fb4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:59.346202 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26acb6d9-7429-4a6c-86ca-0d70b4402fb4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:59.348117 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26acb6d9-7429-4a6c-86ca-0d70b4402fb4 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:59.450718 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-4a474995-8630-425e-bbd8-b1848733a879 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2765a6d16fe46a6960e2949d247eeb5', 'request_id': 'req-4a474995-8630-425e-bbd8-b1848733a879'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:59.521278 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1380/2722] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:51:59 2022] POST /identity/v3/projects => generated 373 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:51:59.525358 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26acb6d9-7429-4a6c-86ca-0d70b4402fb4 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:59.537085 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-4a474995-8630-425e-bbd8-b1848733a879 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2765a6d16fe46a6960e2949d247eeb5', 'request_id': 'req-4a474995-8630-425e-bbd8-b1848733a879'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:59.543507 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-4a474995-8630-425e-bbd8-b1848733a879 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2765a6d16fe46a6960e2949d247eeb5', 'request_id': 'req-4a474995-8630-425e-bbd8-b1848733a879'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:59.555438 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-4a474995-8630-425e-bbd8-b1848733a879 admin admin] The token cache is being invalidate because project e2765a6d16fe46a6960e2949d247eeb5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:51:59.555739 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-4a474995-8630-425e-bbd8-b1848733a879 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:59.556443 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-4a474995-8630-425e-bbd8-b1848733a879 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:51:59.558106 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1343/2723] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:51:59 2022] DELETE /identity/v3/projects/e2765a6d16fe46a6960e2949d247eeb5 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:51:59.563429 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe30b34-bc7c-4f8a-868d-ee379dc2ce1b None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:59.563723 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe30b34-bc7c-4f8a-868d-ee379dc2ce1b None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:59.564010 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe30b34-bc7c-4f8a-868d-ee379dc2ce1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:59.589614 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-1fe30b34-bc7c-4f8a-868d-ee379dc2ce1b None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:51:59.590960 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-1fe30b34-bc7c-4f8a-868d-ee379dc2ce1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:51:59.594832 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26acb6d9-7429-4a6c-86ca-0d70b4402fb4 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:59.599865 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db565717-1d72-4a93-b5d5-7f31834a1e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:59.606190 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1344/2724] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:51:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:51:59.606316 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db565717-1d72-4a93-b5d5-7f31834a1e73 admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:59.606316 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db565717-1d72-4a93-b5d5-7f31834a1e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:59.606316 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db565717-1d72-4a93-b5d5-7f31834a1e73 admin admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:59.606316 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db565717-1d72-4a93-b5d5-7f31834a1e73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:59.606316 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db565717-1d72-4a93-b5d5-7f31834a1e73 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:59.607590 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc2dc2d-4a56-42b6-a6cb-d5f6597f0b3a None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:59.607907 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc2dc2d-4a56-42b6-a6cb-d5f6597f0b3a None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:59.608186 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc2dc2d-4a56-42b6-a6cb-d5f6597f0b3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:59.627755 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-fbc2dc2d-4a56-42b6-a6cb-d5f6597f0b3a None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:51:59.628982 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-fbc2dc2d-4a56-42b6-a6cb-d5f6597f0b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:51:59.639159 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1345/2725] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:51:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:51:59.644075 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc2dc2d-4a56-42b6-a6cb-d5f6597f0b3a None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:59.654313 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc2dc2d-4a56-42b6-a6cb-d5f6597f0b3a None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:59.659919 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e57699-e788-4108-a249-d51a226a4e7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:59.664873 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e57699-e788-4108-a249-d51a226a4e7f admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:59.665269 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e57699-e788-4108-a249-d51a226a4e7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:59.665700 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e57699-e788-4108-a249-d51a226a4e7f admin admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:59.666745 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e57699-e788-4108-a249-d51a226a4e7f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:51:59.667126 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e57699-e788-4108-a249-d51a226a4e7f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:59.668736 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e57699-e788-4108-a249-d51a226a4e7f admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:59.675519 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1381/2726] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:51:59 2022] POST /identity/v3/users => generated 287 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:51:59.676487 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db565717-1d72-4a93-b5d5-7f31834a1e73 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:59.678280 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-db565717-1d72-4a93-b5d5-7f31834a1e73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:51:59.684984 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1346/2727] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jun 7 14:51:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:51:59.685992 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e57699-e788-4108-a249-d51a226a4e7f admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:59.691029 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db565717-1d72-4a93-b5d5-7f31834a1e73 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:59.696238 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e57699-e788-4108-a249-d51a226a4e7f admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:59.697055 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f15e352-e65e-492c-a701-7ae0ef811aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:59.698000 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f15e352-e65e-492c-a701-7ae0ef811aee admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:59.698367 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f15e352-e65e-492c-a701-7ae0ef811aee admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:59.698724 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f15e352-e65e-492c-a701-7ae0ef811aee admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:59.699547 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f15e352-e65e-492c-a701-7ae0ef811aee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:59.701259 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f15e352-e65e-492c-a701-7ae0ef811aee admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:59.702137 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57be586d-cbca-4b53-bf7c-6d0395dfb722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:59.702976 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57be586d-cbca-4b53-bf7c-6d0395dfb722 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:59.703271 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57be586d-cbca-4b53-bf7c-6d0395dfb722 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:59.703546 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57be586d-cbca-4b53-bf7c-6d0395dfb722 admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:59.704235 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57be586d-cbca-4b53-bf7c-6d0395dfb722 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:59.706121 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57be586d-cbca-4b53-bf7c-6d0395dfb722 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:59.712931 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1382/2728] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:51:59 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:51:59.720910 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f15e352-e65e-492c-a701-7ae0ef811aee admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:59.730598 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f15e352-e65e-492c-a701-7ae0ef811aee admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:59.736289 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c925a7-2c7c-4fa6-81fa-98accffb71e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:59.736984 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c925a7-2c7c-4fa6-81fa-98accffb71e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:59.737271 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c925a7-2c7c-4fa6-81fa-98accffb71e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:59.737544 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c925a7-2c7c-4fa6-81fa-98accffb71e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:59.738267 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c925a7-2c7c-4fa6-81fa-98accffb71e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:59.739857 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c925a7-2c7c-4fa6-81fa-98accffb71e0 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:59.750842 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1383/2729] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:51:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 14:51:59.754890 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c925a7-2c7c-4fa6-81fa-98accffb71e0 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:59.764953 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c925a7-2c7c-4fa6-81fa-98accffb71e0 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:59.771027 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d4a9dd-8b7e-4b15-bcba-8905d682c059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:59.779914 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d4a9dd-8b7e-4b15-bcba-8905d682c059 admin admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:59.779914 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d4a9dd-8b7e-4b15-bcba-8905d682c059 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:59.779914 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d4a9dd-8b7e-4b15-bcba-8905d682c059 admin admin] PATH_INFO: `/v3/projects/fc7a18c877864f6a91cafd369105bcfa/users/be9dec3c8efc4ab580ab02d3fd6b3e4c/roles/3771341b3be641139e752aab690397a4` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:59.790216 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d4a9dd-8b7e-4b15-bcba-8905d682c059 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc7a18c877864f6a91cafd369105bcfa, user_id=be9dec3c8efc4ab580ab02d3fd6b3e4c, role_id=3771341b3be641139e752aab690397a4)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:59.792879 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d4a9dd-8b7e-4b15-bcba-8905d682c059 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:51:59.831534 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1347/2730] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:51:59 2022] POST /identity/v3/projects => generated 399 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:51:59.835566 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57be586d-cbca-4b53-bf7c-6d0395dfb722 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:51:59.883347 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:51:59.886253 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1384/2731] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:51:59 2022] PUT /identity/v3/projects/fc7a18c877864f6a91cafd369105bcfa/users/be9dec3c8efc4ab580ab02d3fd6b3e4c/roles/3771341b3be641139e752aab690397a4 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:51:59.896722 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0675cc48-bb91-42a1-8da4-242bc8dc38a8 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:59.897133 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0675cc48-bb91-42a1-8da4-242bc8dc38a8 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:59.897482 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0675cc48-bb91-42a1-8da4-242bc8dc38a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:59.898448 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57be586d-cbca-4b53-bf7c-6d0395dfb722 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:51:59.927379 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-0675cc48-bb91-42a1-8da4-242bc8dc38a8 None None] MFA Rules not processed for user `be9dec3c8efc4ab580ab02d3fd6b3e4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:51:59.928761 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-0675cc48-bb91-42a1-8da4-242bc8dc38a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:51:59.960737 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64208b46-ebfc-4dde-82d5-715ef9aa36c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:51:59.992012 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64208b46-ebfc-4dde-82d5-715ef9aa36c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:51:59.992012 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64208b46-ebfc-4dde-82d5-715ef9aa36c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:51:59.992012 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64208b46-ebfc-4dde-82d5-715ef9aa36c2 admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:51:59.992012 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64208b46-ebfc-4dde-82d5-715ef9aa36c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:51:59.992012 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64208b46-ebfc-4dde-82d5-715ef9aa36c2 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:00.010806 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1385/2732] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:51:59 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:52:00.024019 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0675cc48-bb91-42a1-8da4-242bc8dc38a8 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:00.025808 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-0675cc48-bb91-42a1-8da4-242bc8dc38a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:52:00.038440 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:52:00.104510 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0675cc48-bb91-42a1-8da4-242bc8dc38a8 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:00.110455 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e541840c-d484-4c4c-858b-fb01801c661b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e857db4902e42989d0d040bbc8517c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:00.111102 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e541840c-d484-4c4c-858b-fb01801c661b service nova] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:00.111386 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e541840c-d484-4c4c-858b-fb01801c661b service nova] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:00.111662 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e541840c-d484-4c4c-858b-fb01801c661b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:00.119179 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e541840c-d484-4c4c-858b-fb01801c661b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:00.120795 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e541840c-d484-4c4c-858b-fb01801c661b service nova] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:00.129789 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1386/2733] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:52:00 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 14:52:00.188200 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1348/2734] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:51:59 2022] POST /identity/v3/users => generated 300 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:52:00.190206 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e541840c-d484-4c4c-858b-fb01801c661b service nova] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:00.199219 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e541840c-d484-4c4c-858b-fb01801c661b service nova] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:00.205456 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd186947-5cfa-4f2c-a88a-8a0b3b4e87c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:00.206149 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd186947-5cfa-4f2c-a88a-8a0b3b4e87c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:00.206431 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd186947-5cfa-4f2c-a88a-8a0b3b4e87c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:00.206707 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd186947-5cfa-4f2c-a88a-8a0b3b4e87c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:00.207431 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd186947-5cfa-4f2c-a88a-8a0b3b4e87c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:00.209053 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd186947-5cfa-4f2c-a88a-8a0b3b4e87c9 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:00.222495 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1387/2735] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:52:00 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:52:00.226669 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64208b46-ebfc-4dde-82d5-715ef9aa36c2 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:00.239939 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64208b46-ebfc-4dde-82d5-715ef9aa36c2 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:00.246166 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96964e4f-0ce7-45cd-88ce-4dcd3cfc87d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:00.246843 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96964e4f-0ce7-45cd-88ce-4dcd3cfc87d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:00.247132 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96964e4f-0ce7-45cd-88ce-4dcd3cfc87d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:00.247411 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96964e4f-0ce7-45cd-88ce-4dcd3cfc87d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:00.248142 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96964e4f-0ce7-45cd-88ce-4dcd3cfc87d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:00.257175 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96964e4f-0ce7-45cd-88ce-4dcd3cfc87d9 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:00.266459 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1349/2736] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:52:00 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:52:00.267627 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd186947-5cfa-4f2c-a88a-8a0b3b4e87c9 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:00.278176 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd186947-5cfa-4f2c-a88a-8a0b3b4e87c9 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:00.285540 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee38d84-2950-48e1-ba3a-b345ab6617e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:00.286442 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee38d84-2950-48e1-ba3a-b345ab6617e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:00.286728 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee38d84-2950-48e1-ba3a-b345ab6617e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:00.287005 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee38d84-2950-48e1-ba3a-b345ab6617e2 admin admin] PATH_INFO: `/v3/projects/3a03a2b8deed46b9a96ab27eed222051/users/bd5cf2faa24c4e14a86081cd5b004a86/roles/3771341b3be641139e752aab690397a4` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:00.307213 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee38d84-2950-48e1-ba3a-b345ab6617e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a03a2b8deed46b9a96ab27eed222051, user_id=bd5cf2faa24c4e14a86081cd5b004a86, role_id=3771341b3be641139e752aab690397a4)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:00.309864 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee38d84-2950-48e1-ba3a-b345ab6617e2 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:00.347820 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96964e4f-0ce7-45cd-88ce-4dcd3cfc87d9 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:00.358321 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96964e4f-0ce7-45cd-88ce-4dcd3cfc87d9 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:00.363124 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5956750-f206-4928-8c07-274741f7ffde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:00.364027 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5956750-f206-4928-8c07-274741f7ffde admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:00.364366 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5956750-f206-4928-8c07-274741f7ffde admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:00.364679 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5956750-f206-4928-8c07-274741f7ffde admin admin] PATH_INFO: `/v3/users/be9dec3c8efc4ab580ab02d3fd6b3e4c` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:00.365917 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5956750-f206-4928-8c07-274741f7ffde admin admin] RBAC: Authorizing `identity:delete_user(user_id=be9dec3c8efc4ab580ab02d3fd6b3e4c)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:00.367819 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5956750-f206-4928-8c07-274741f7ffde admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:00.393253 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1388/2737] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:52:00 2022] PUT /identity/v3/projects/3a03a2b8deed46b9a96ab27eed222051/users/bd5cf2faa24c4e14a86081cd5b004a86/roles/3771341b3be641139e752aab690397a4 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:52:00.403053 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871eccd8-0a8b-4923-90b8-d1801dad0b71 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:00.405911 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871eccd8-0a8b-4923-90b8-d1801dad0b71 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:00.405911 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871eccd8-0a8b-4923-90b8-d1801dad0b71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:00.430162 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-871eccd8-0a8b-4923-90b8-d1801dad0b71 None None] MFA Rules not processed for user `bd5cf2faa24c4e14a86081cd5b004a86`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:52:00.432043 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-871eccd8-0a8b-4923-90b8-d1801dad0b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:52:00.495361 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1389/2738] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:52:00 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:52:00.507319 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871eccd8-0a8b-4923-90b8-d1801dad0b71 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:00.508750 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-871eccd8-0a8b-4923-90b8-d1801dad0b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:52:00.562319 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:52:00.564273 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a5956750-f206-4928-8c07-274741f7ffde admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be9dec3c8efc4ab580ab02d3fd6b3e4c'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:00.564886 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a5956750-f206-4928-8c07-274741f7ffde admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be9dec3c8efc4ab580ab02d3fd6b3e4c'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:00.565623 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a5956750-f206-4928-8c07-274741f7ffde admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be9dec3c8efc4ab580ab02d3fd6b3e4c'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:00.567679 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871eccd8-0a8b-4923-90b8-d1801dad0b71 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:00.571857 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b1f42b-c701-4f3d-9df1-1eb457608c34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d81f597cb44c56a966537089bcce4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:00.572572 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b1f42b-c701-4f3d-9df1-1eb457608c34 service cinder] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:00.572859 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b1f42b-c701-4f3d-9df1-1eb457608c34 service cinder] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:00.573138 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b1f42b-c701-4f3d-9df1-1eb457608c34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:00.573255 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a5956750-f206-4928-8c07-274741f7ffde admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be9dec3c8efc4ab580ab02d3fd6b3e4c'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:00.574959 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-53b1f42b-c701-4f3d-9df1-1eb457608c34 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:52:00.586710 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a5956750-f206-4928-8c07-274741f7ffde admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be9dec3c8efc4ab580ab02d3fd6b3e4c'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:00.586940 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b1f42b-c701-4f3d-9df1-1eb457608c34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:00.588313 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b1f42b-c701-4f3d-9df1-1eb457608c34 service cinder] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:00.597820 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1390/2739] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:52:00 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 14:52:00.652283 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1350/2740] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:52:00 2022] DELETE /identity/v3/users/be9dec3c8efc4ab580ab02d3fd6b3e4c => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:52:00.656951 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6935e621-baf7-4e14-916e-2aa600428252 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:00.662954 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6935e621-baf7-4e14-916e-2aa600428252 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:00.662954 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6935e621-baf7-4e14-916e-2aa600428252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:00.680999 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-6935e621-baf7-4e14-916e-2aa600428252 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:52:00.680999 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-6935e621-baf7-4e14-916e-2aa600428252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:52:00.756554 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1391/2741] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:52:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:52:00.768914 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5956750-f206-4928-8c07-274741f7ffde admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:00.770566 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-a5956750-f206-4928-8c07-274741f7ffde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:52:00.825019 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:52:00.837107 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5956750-f206-4928-8c07-274741f7ffde admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:00.841163 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456815a5-5daf-4c08-8c07-a4411c883516 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:00.841841 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456815a5-5daf-4c08-8c07-a4411c883516 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:00.842129 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456815a5-5daf-4c08-8c07-a4411c883516 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:00.842411 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456815a5-5daf-4c08-8c07-a4411c883516 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:00.853062 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456815a5-5daf-4c08-8c07-a4411c883516 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:00.854524 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456815a5-5daf-4c08-8c07-a4411c883516 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:00.866572 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1351/2742] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:52:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:52:01.356342 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6935e621-baf7-4e14-916e-2aa600428252 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:01.358088 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-6935e621-baf7-4e14-916e-2aa600428252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:52:01.371077 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6935e621-baf7-4e14-916e-2aa600428252 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:01.375978 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fb5113-bc45-4667-a445-99c3d55d2844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:01.376888 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fb5113-bc45-4667-a445-99c3d55d2844 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:01.377175 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fb5113-bc45-4667-a445-99c3d55d2844 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:01.377455 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fb5113-bc45-4667-a445-99c3d55d2844 admin admin] PATH_INFO: `/v3/projects/fc7a18c877864f6a91cafd369105bcfa` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:01.378658 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fb5113-bc45-4667-a445-99c3d55d2844 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc7a18c877864f6a91cafd369105bcfa)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:01.381420 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fb5113-bc45-4667-a445-99c3d55d2844 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:01.475287 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-30fb5113-bc45-4667-a445-99c3d55d2844 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc7a18c877864f6a91cafd369105bcfa', 'request_id': 'req-30fb5113-bc45-4667-a445-99c3d55d2844'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:01.537120 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-30fb5113-bc45-4667-a445-99c3d55d2844 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc7a18c877864f6a91cafd369105bcfa', 'request_id': 'req-30fb5113-bc45-4667-a445-99c3d55d2844'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:01.543255 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-30fb5113-bc45-4667-a445-99c3d55d2844 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc7a18c877864f6a91cafd369105bcfa', 'request_id': 'req-30fb5113-bc45-4667-a445-99c3d55d2844'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:01.554812 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-30fb5113-bc45-4667-a445-99c3d55d2844 admin admin] The token cache is being invalidate because project fc7a18c877864f6a91cafd369105bcfa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:52:01.555102 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-30fb5113-bc45-4667-a445-99c3d55d2844 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:01.555748 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-30fb5113-bc45-4667-a445-99c3d55d2844 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:01.557313 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1392/2743] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:52:01 2022] DELETE /identity/v3/projects/fc7a18c877864f6a91cafd369105bcfa => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:52:01.562297 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2dc103-8928-45f8-bf61-fd2e99aa0a0e None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:01.562772 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2dc103-8928-45f8-bf61-fd2e99aa0a0e None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:01.562896 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2dc103-8928-45f8-bf61-fd2e99aa0a0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:01.581494 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-2a2dc103-8928-45f8-bf61-fd2e99aa0a0e None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:52:01.582893 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-2a2dc103-8928-45f8-bf61-fd2e99aa0a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:52:01.644906 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1352/2744] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:52:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:52:01.653086 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2a4d1f-a73c-4116-838c-2e22382de821 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:01.653263 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2a4d1f-a73c-4116-838c-2e22382de821 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:01.653442 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2a4d1f-a73c-4116-838c-2e22382de821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:01.672962 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-4a2a4d1f-a73c-4116-838c-2e22382de821 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:52:01.674188 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-4a2a4d1f-a73c-4116-838c-2e22382de821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:52:01.682977 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1393/2745] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:52:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:52:01.687551 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2dc103-8928-45f8-bf61-fd2e99aa0a0e None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:01.698098 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2dc103-8928-45f8-bf61-fd2e99aa0a0e None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:01.703480 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7499ac2c-79bb-4074-ba41-5a7218b77c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:01.704148 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7499ac2c-79bb-4074-ba41-5a7218b77c50 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:01.704465 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7499ac2c-79bb-4074-ba41-5a7218b77c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:01.704768 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7499ac2c-79bb-4074-ba41-5a7218b77c50 admin admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:01.705641 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7499ac2c-79bb-4074-ba41-5a7218b77c50 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:52:01.705960 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7499ac2c-79bb-4074-ba41-5a7218b77c50 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:01.707563 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7499ac2c-79bb-4074-ba41-5a7218b77c50 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:01.723740 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1353/2746] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jun 7 14:52:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:52:01.724847 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2a4d1f-a73c-4116-838c-2e22382de821 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:01.734161 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2a4d1f-a73c-4116-838c-2e22382de821 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:01.739018 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5f41c1-b570-4209-87b2-76b15c0661f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:01.739671 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5f41c1-b570-4209-87b2-76b15c0661f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:01.739960 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5f41c1-b570-4209-87b2-76b15c0661f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:01.740241 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5f41c1-b570-4209-87b2-76b15c0661f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:01.740998 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5f41c1-b570-4209-87b2-76b15c0661f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:01.742883 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5f41c1-b570-4209-87b2-76b15c0661f0 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:01.837863 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1394/2747] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:52:01 2022] POST /identity/v3/projects => generated 367 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:52:01.845384 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7499ac2c-79bb-4074-ba41-5a7218b77c50 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:01.855813 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7499ac2c-79bb-4074-ba41-5a7218b77c50 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:01.912047 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdcfc96-c255-4389-a1ca-4b65673d229f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:01.912735 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdcfc96-c255-4389-a1ca-4b65673d229f admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:01.913017 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdcfc96-c255-4389-a1ca-4b65673d229f admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:01.913291 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdcfc96-c255-4389-a1ca-4b65673d229f admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:01.913985 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdcfc96-c255-4389-a1ca-4b65673d229f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:01.915922 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdcfc96-c255-4389-a1ca-4b65673d229f admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:01.983485 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1354/2748] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:52:01 2022] POST /identity/v3/users => generated 284 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:52:01.987883 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5f41c1-b570-4209-87b2-76b15c0661f0 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:01.998508 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5f41c1-b570-4209-87b2-76b15c0661f0 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:02.003946 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02ba435-1cb8-49c4-b1c2-251569cac182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:02.004642 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02ba435-1cb8-49c4-b1c2-251569cac182 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:02.004929 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02ba435-1cb8-49c4-b1c2-251569cac182 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:02.005209 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02ba435-1cb8-49c4-b1c2-251569cac182 admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:02.005940 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02ba435-1cb8-49c4-b1c2-251569cac182 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:02.007549 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02ba435-1cb8-49c4-b1c2-251569cac182 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:02.021026 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1395/2749] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:52:01 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:52:02.025481 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdcfc96-c255-4389-a1ca-4b65673d229f admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:02.036666 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdcfc96-c255-4389-a1ca-4b65673d229f admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:02.041615 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7498af7f-68ee-4e4c-9250-fa75646a9f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:02.042277 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7498af7f-68ee-4e4c-9250-fa75646a9f27 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:02.042562 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7498af7f-68ee-4e4c-9250-fa75646a9f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:02.042842 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7498af7f-68ee-4e4c-9250-fa75646a9f27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:02.043568 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7498af7f-68ee-4e4c-9250-fa75646a9f27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:02.045330 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7498af7f-68ee-4e4c-9250-fa75646a9f27 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:02.056793 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1355/2750] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:52:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:52:02.060938 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02ba435-1cb8-49c4-b1c2-251569cac182 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:02.071994 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02ba435-1cb8-49c4-b1c2-251569cac182 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:02.078048 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e31b6b-311d-49e6-b027-5d8169b4c527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:02.079107 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e31b6b-311d-49e6-b027-5d8169b4c527 admin admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:02.079390 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e31b6b-311d-49e6-b027-5d8169b4c527 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:02.079665 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e31b6b-311d-49e6-b027-5d8169b4c527 admin admin] PATH_INFO: `/v3/projects/ad5ce3a7c9be42ac802ee2c813a304a6/users/19c7e22fcd67457cb287e35e30b0e08b/roles/3771341b3be641139e752aab690397a4` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:02.096040 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e31b6b-311d-49e6-b027-5d8169b4c527 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad5ce3a7c9be42ac802ee2c813a304a6, user_id=19c7e22fcd67457cb287e35e30b0e08b, role_id=3771341b3be641139e752aab690397a4)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:02.098870 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e31b6b-311d-49e6-b027-5d8169b4c527 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:02.183371 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1396/2751] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:52:02 2022] PUT /identity/v3/projects/ad5ce3a7c9be42ac802ee2c813a304a6/users/19c7e22fcd67457cb287e35e30b0e08b/roles/3771341b3be641139e752aab690397a4 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:52:02.195525 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7def376d-0c98-4cf1-94ac-1337fa3cfb05 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:02.195817 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7def376d-0c98-4cf1-94ac-1337fa3cfb05 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:02.196101 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7def376d-0c98-4cf1-94ac-1337fa3cfb05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:02.224350 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-7def376d-0c98-4cf1-94ac-1337fa3cfb05 None None] MFA Rules not processed for user `19c7e22fcd67457cb287e35e30b0e08b`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:52:02.225613 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-7def376d-0c98-4cf1-94ac-1337fa3cfb05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:52:02.269955 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:52:02.292711 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1356/2752] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:52:02 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:52:02.297785 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e31b6b-311d-49e6-b027-5d8169b4c527 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:02.308120 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:52:02.358923 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e31b6b-311d-49e6-b027-5d8169b4c527 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:02.363766 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dc3639-8430-45d8-aa39-087b59f6be59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:02.364455 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dc3639-8430-45d8-aa39-087b59f6be59 admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:02.364800 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dc3639-8430-45d8-aa39-087b59f6be59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:02.365026 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dc3639-8430-45d8-aa39-087b59f6be59 admin admin] PATH_INFO: `/v3/projects` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:02.365731 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dc3639-8430-45d8-aa39-087b59f6be59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:02.367626 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dc3639-8430-45d8-aa39-087b59f6be59 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:02.446983 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1397/2753] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:52:02 2022] POST /identity/v3/projects => generated 367 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:52:02.451187 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7def376d-0c98-4cf1-94ac-1337fa3cfb05 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:02.462571 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7def376d-0c98-4cf1-94ac-1337fa3cfb05 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:02.518707 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f360af9-966a-44ed-99f5-55cd2dffa68e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:02.519387 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f360af9-966a-44ed-99f5-55cd2dffa68e admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:02.519680 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f360af9-966a-44ed-99f5-55cd2dffa68e admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:02.520126 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f360af9-966a-44ed-99f5-55cd2dffa68e admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:02.520865 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f360af9-966a-44ed-99f5-55cd2dffa68e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:02.522775 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f360af9-966a-44ed-99f5-55cd2dffa68e admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:02.629289 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1357/2754] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:52:02 2022] POST /identity/v3/users => generated 290 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:52:02.633118 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dc3639-8430-45d8-aa39-087b59f6be59 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:02.644313 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dc3639-8430-45d8-aa39-087b59f6be59 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:02.649031 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2b841d-41d2-4e16-8a44-21412831b011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:02.649824 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2b841d-41d2-4e16-8a44-21412831b011 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:02.650053 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2b841d-41d2-4e16-8a44-21412831b011 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:02.650331 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2b841d-41d2-4e16-8a44-21412831b011 admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:02.651067 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2b841d-41d2-4e16-8a44-21412831b011 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:02.652951 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2b841d-41d2-4e16-8a44-21412831b011 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:02.664745 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1398/2755] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:52:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:52:02.669101 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f360af9-966a-44ed-99f5-55cd2dffa68e admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:02.687637 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f360af9-966a-44ed-99f5-55cd2dffa68e admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:02.693142 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f2cf4e-4c9e-4af7-adc1-fce78b96935a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:02.694067 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f2cf4e-4c9e-4af7-adc1-fce78b96935a admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:02.694353 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f2cf4e-4c9e-4af7-adc1-fce78b96935a admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:02.694628 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f2cf4e-4c9e-4af7-adc1-fce78b96935a admin admin] PATH_INFO: `/v3/projects/2ca2e7cd0350485aa6e3f6e7005222c4/users/0e90ea3a893c40d691090fdc3ab9c767/roles/9baff2b18f2649f7b757ff2e1f63c853` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:02.710236 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f2cf4e-4c9e-4af7-adc1-fce78b96935a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ca2e7cd0350485aa6e3f6e7005222c4, user_id=0e90ea3a893c40d691090fdc3ab9c767, role_id=9baff2b18f2649f7b757ff2e1f63c853)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:02.712914 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f2cf4e-4c9e-4af7-adc1-fce78b96935a admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:02.829947 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1358/2756] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:52:02 2022] PUT /identity/v3/projects/2ca2e7cd0350485aa6e3f6e7005222c4/users/0e90ea3a893c40d691090fdc3ab9c767/roles/9baff2b18f2649f7b757ff2e1f63c853 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:52:02.834123 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2b841d-41d2-4e16-8a44-21412831b011 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:02.844566 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2b841d-41d2-4e16-8a44-21412831b011 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:02.902778 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c214b527-3333-4196-a9c6-23c4f9a24a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:02.903327 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c214b527-3333-4196-a9c6-23c4f9a24a61 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:02.903616 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c214b527-3333-4196-a9c6-23c4f9a24a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:02.903897 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c214b527-3333-4196-a9c6-23c4f9a24a61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:02.904656 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c214b527-3333-4196-a9c6-23c4f9a24a61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:02.906267 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c214b527-3333-4196-a9c6-23c4f9a24a61 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:02.916231 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1399/2757] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:52:02 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:52:02.920733 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f2cf4e-4c9e-4af7-adc1-fce78b96935a admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:02.931337 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f2cf4e-4c9e-4af7-adc1-fce78b96935a admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:02.936793 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dff8269-b73b-4e1a-b72b-c505e6fe8ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:02.937711 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dff8269-b73b-4e1a-b72b-c505e6fe8ea3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:02.937998 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dff8269-b73b-4e1a-b72b-c505e6fe8ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:02.938276 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dff8269-b73b-4e1a-b72b-c505e6fe8ea3 admin admin] PATH_INFO: `/v3/projects/2ca2e7cd0350485aa6e3f6e7005222c4/users/0e90ea3a893c40d691090fdc3ab9c767/roles/9baff2b18f2649f7b757ff2e1f63c853` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:02.940468 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dff8269-b73b-4e1a-b72b-c505e6fe8ea3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ca2e7cd0350485aa6e3f6e7005222c4, user_id=0e90ea3a893c40d691090fdc3ab9c767, role_id=9baff2b18f2649f7b757ff2e1f63c853)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:02.943263 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dff8269-b73b-4e1a-b72b-c505e6fe8ea3 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:02.955279 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1359/2758] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:52:02 2022] PUT /identity/v3/projects/2ca2e7cd0350485aa6e3f6e7005222c4/users/0e90ea3a893c40d691090fdc3ab9c767/roles/9baff2b18f2649f7b757ff2e1f63c853 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:52:02.966424 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650eea8c-0d15-4709-935a-4179a8cafc08 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:02.966797 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650eea8c-0d15-4709-935a-4179a8cafc08 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:02.967042 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650eea8c-0d15-4709-935a-4179a8cafc08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:02.993638 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-650eea8c-0d15-4709-935a-4179a8cafc08 None None] MFA Rules not processed for user `0e90ea3a893c40d691090fdc3ab9c767`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:52:02.994846 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-650eea8c-0d15-4709-935a-4179a8cafc08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:52:03.066118 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1400/2759] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:52:02 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:52:03.076977 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dff8269-b73b-4e1a-b72b-c505e6fe8ea3 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:03.078852 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-5dff8269-b73b-4e1a-b72b-c505e6fe8ea3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:52:03.145181 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dff8269-b73b-4e1a-b72b-c505e6fe8ea3 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:03.148988 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d9c6e8-95f2-469e-9f9f-458329c42d11 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d81f597cb44c56a966537089bcce4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:03.149647 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d9c6e8-95f2-469e-9f9f-458329c42d11 service cinder] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:03.149933 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d9c6e8-95f2-469e-9f9f-458329c42d11 service cinder] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:03.150215 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d9c6e8-95f2-469e-9f9f-458329c42d11 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:03.160271 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d9c6e8-95f2-469e-9f9f-458329c42d11 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:03.167444 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d9c6e8-95f2-469e-9f9f-458329c42d11 service cinder] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:03.172341 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1360/2760] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:52:03 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:52:06.131825 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650eea8c-0d15-4709-935a-4179a8cafc08 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:06.137404 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:52:06.197059 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650eea8c-0d15-4709-935a-4179a8cafc08 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:06.202182 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96f229d-6ede-4a48-81e1-0233de100a3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:06.203092 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96f229d-6ede-4a48-81e1-0233de100a3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:06.203406 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96f229d-6ede-4a48-81e1-0233de100a3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:06.203688 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96f229d-6ede-4a48-81e1-0233de100a3e admin admin] PATH_INFO: `/v3/users/19c7e22fcd67457cb287e35e30b0e08b` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:06.204934 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96f229d-6ede-4a48-81e1-0233de100a3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=19c7e22fcd67457cb287e35e30b0e08b)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:06.206838 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96f229d-6ede-4a48-81e1-0233de100a3e admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:06.411469 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-e96f229d-6ede-4a48-81e1-0233de100a3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19c7e22fcd67457cb287e35e30b0e08b'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:06.411469 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-e96f229d-6ede-4a48-81e1-0233de100a3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19c7e22fcd67457cb287e35e30b0e08b'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:06.411469 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-e96f229d-6ede-4a48-81e1-0233de100a3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19c7e22fcd67457cb287e35e30b0e08b'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:06.417885 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-e96f229d-6ede-4a48-81e1-0233de100a3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19c7e22fcd67457cb287e35e30b0e08b'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:06.430560 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-e96f229d-6ede-4a48-81e1-0233de100a3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19c7e22fcd67457cb287e35e30b0e08b'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:06.501213 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1401/2761] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:52:06 2022] DELETE /identity/v3/users/19c7e22fcd67457cb287e35e30b0e08b => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:52:06.505644 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc34b0df-ed2f-412d-b2b8-a0ab7de513c9 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:06.508812 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc34b0df-ed2f-412d-b2b8-a0ab7de513c9 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:06.508812 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc34b0df-ed2f-412d-b2b8-a0ab7de513c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:06.523645 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-fc34b0df-ed2f-412d-b2b8-a0ab7de513c9 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:52:06.524894 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-fc34b0df-ed2f-412d-b2b8-a0ab7de513c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:52:06.557328 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:52:06.582815 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1361/2762] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:52:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:52:06.592667 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96f229d-6ede-4a48-81e1-0233de100a3e admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:06.594619 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-e96f229d-6ede-4a48-81e1-0233de100a3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:52:06.653188 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96f229d-6ede-4a48-81e1-0233de100a3e admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:06.657325 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0cde8e-8140-4411-b22b-edcba09b74f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:06.657975 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0cde8e-8140-4411-b22b-edcba09b74f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:06.658260 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0cde8e-8140-4411-b22b-edcba09b74f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:06.658538 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0cde8e-8140-4411-b22b-edcba09b74f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:06.667801 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0cde8e-8140-4411-b22b-edcba09b74f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:06.669268 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0cde8e-8140-4411-b22b-edcba09b74f5 service neutron] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:06.679542 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1402/2763] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:52:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 14:52:07.110298 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc34b0df-ed2f-412d-b2b8-a0ab7de513c9 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:07.114919 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-fc34b0df-ed2f-412d-b2b8-a0ab7de513c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:52:07.124051 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc34b0df-ed2f-412d-b2b8-a0ab7de513c9 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:07.129561 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b35d25-7e0f-4cbf-b2f5-a9a21a453d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:07.130480 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b35d25-7e0f-4cbf-b2f5-a9a21a453d9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:07.130770 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b35d25-7e0f-4cbf-b2f5-a9a21a453d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:07.131049 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b35d25-7e0f-4cbf-b2f5-a9a21a453d9c admin admin] PATH_INFO: `/v3/projects/ad5ce3a7c9be42ac802ee2c813a304a6` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:07.132250 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b35d25-7e0f-4cbf-b2f5-a9a21a453d9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad5ce3a7c9be42ac802ee2c813a304a6)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:07.135147 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b35d25-7e0f-4cbf-b2f5-a9a21a453d9c admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:07.235777 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-43b35d25-7e0f-4cbf-b2f5-a9a21a453d9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad5ce3a7c9be42ac802ee2c813a304a6', 'request_id': 'req-43b35d25-7e0f-4cbf-b2f5-a9a21a453d9c'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:07.300980 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-43b35d25-7e0f-4cbf-b2f5-a9a21a453d9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad5ce3a7c9be42ac802ee2c813a304a6', 'request_id': 'req-43b35d25-7e0f-4cbf-b2f5-a9a21a453d9c'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:07.307279 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-43b35d25-7e0f-4cbf-b2f5-a9a21a453d9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad5ce3a7c9be42ac802ee2c813a304a6', 'request_id': 'req-43b35d25-7e0f-4cbf-b2f5-a9a21a453d9c'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:07.319387 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-43b35d25-7e0f-4cbf-b2f5-a9a21a453d9c admin admin] The token cache is being invalidate because project ad5ce3a7c9be42ac802ee2c813a304a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:52:07.319681 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-43b35d25-7e0f-4cbf-b2f5-a9a21a453d9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:07.320350 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-43b35d25-7e0f-4cbf-b2f5-a9a21a453d9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:07.321985 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1362/2764] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:52:07 2022] DELETE /identity/v3/projects/ad5ce3a7c9be42ac802ee2c813a304a6 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:52:07.326233 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0cde8e-8140-4411-b22b-edcba09b74f5 service neutron] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:07.339561 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0cde8e-8140-4411-b22b-edcba09b74f5 service neutron] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:07.363203 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:52:07.396920 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39291f0a-bc28-4ed6-8f6b-9e1c6ad10bde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:07.397843 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39291f0a-bc28-4ed6-8f6b-9e1c6ad10bde admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:07.398131 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39291f0a-bc28-4ed6-8f6b-9e1c6ad10bde admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:07.398412 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39291f0a-bc28-4ed6-8f6b-9e1c6ad10bde admin admin] PATH_INFO: `/v3/users/0e90ea3a893c40d691090fdc3ab9c767` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:07.399811 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39291f0a-bc28-4ed6-8f6b-9e1c6ad10bde admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e90ea3a893c40d691090fdc3ab9c767)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:07.402194 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39291f0a-bc28-4ed6-8f6b-9e1c6ad10bde admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:07.579401 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-39291f0a-bc28-4ed6-8f6b-9e1c6ad10bde admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e90ea3a893c40d691090fdc3ab9c767'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:07.580089 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-39291f0a-bc28-4ed6-8f6b-9e1c6ad10bde admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e90ea3a893c40d691090fdc3ab9c767'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:07.580708 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-39291f0a-bc28-4ed6-8f6b-9e1c6ad10bde admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e90ea3a893c40d691090fdc3ab9c767'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:07.587607 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-39291f0a-bc28-4ed6-8f6b-9e1c6ad10bde admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e90ea3a893c40d691090fdc3ab9c767'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:07.599972 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-39291f0a-bc28-4ed6-8f6b-9e1c6ad10bde admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e90ea3a893c40d691090fdc3ab9c767'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:07.686216 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1403/2765] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:52:07 2022] DELETE /identity/v3/users/0e90ea3a893c40d691090fdc3ab9c767 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:52:08.099007 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b35d25-7e0f-4cbf-b2f5-a9a21a453d9c admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:08.100480 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-43b35d25-7e0f-4cbf-b2f5-a9a21a453d9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:52:08.114494 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b35d25-7e0f-4cbf-b2f5-a9a21a453d9c admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:08.173165 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1219943-e247-4148-87ff-f88422d56bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:08.174035 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1219943-e247-4148-87ff-f88422d56bfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:08.174319 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1219943-e247-4148-87ff-f88422d56bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:08.174605 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1219943-e247-4148-87ff-f88422d56bfd admin admin] PATH_INFO: `/v3/projects/2ca2e7cd0350485aa6e3f6e7005222c4` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:08.175805 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1219943-e247-4148-87ff-f88422d56bfd admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ca2e7cd0350485aa6e3f6e7005222c4)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:08.177796 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1219943-e247-4148-87ff-f88422d56bfd admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:08.286217 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-e1219943-e247-4148-87ff-f88422d56bfd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ca2e7cd0350485aa6e3f6e7005222c4', 'request_id': 'req-e1219943-e247-4148-87ff-f88422d56bfd'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:08.354466 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-e1219943-e247-4148-87ff-f88422d56bfd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ca2e7cd0350485aa6e3f6e7005222c4', 'request_id': 'req-e1219943-e247-4148-87ff-f88422d56bfd'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:08.360012 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:52:08.361291 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-e1219943-e247-4148-87ff-f88422d56bfd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ca2e7cd0350485aa6e3f6e7005222c4', 'request_id': 'req-e1219943-e247-4148-87ff-f88422d56bfd'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:08.373597 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-e1219943-e247-4148-87ff-f88422d56bfd admin admin] The token cache is being invalidate because project 2ca2e7cd0350485aa6e3f6e7005222c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:52:08.373909 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-e1219943-e247-4148-87ff-f88422d56bfd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:08.374561 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-e1219943-e247-4148-87ff-f88422d56bfd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:08.376123 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1363/2766] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:52:08 2022] DELETE /identity/v3/projects/2ca2e7cd0350485aa6e3f6e7005222c4 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:52:08.381563 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b6de8e-aab6-4525-b8a1-352ec30d9d3f None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:08.382241 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b6de8e-aab6-4525-b8a1-352ec30d9d3f None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:08.382241 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b6de8e-aab6-4525-b8a1-352ec30d9d3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:08.402423 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-f5b6de8e-aab6-4525-b8a1-352ec30d9d3f None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:52:08.403632 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-f5b6de8e-aab6-4525-b8a1-352ec30d9d3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:52:08.465586 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1404/2767] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:52:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:52:08.474864 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e57947-b6df-4a4c-8074-ae8a97b9056d None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:08.475156 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e57947-b6df-4a4c-8074-ae8a97b9056d None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:08.475437 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e57947-b6df-4a4c-8074-ae8a97b9056d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:08.493921 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-c9e57947-b6df-4a4c-8074-ae8a97b9056d None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:52:08.495136 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-c9e57947-b6df-4a4c-8074-ae8a97b9056d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:52:08.503255 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1364/2768] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:52:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:52:08.507653 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b6de8e-aab6-4525-b8a1-352ec30d9d3f None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:08.518180 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b6de8e-aab6-4525-b8a1-352ec30d9d3f None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:08.522927 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca97558-4482-4abd-9efd-a563bb3c8b69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:08.523623 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca97558-4482-4abd-9efd-a563bb3c8b69 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:08.523907 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca97558-4482-4abd-9efd-a563bb3c8b69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:08.524212 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca97558-4482-4abd-9efd-a563bb3c8b69 admin admin] PATH_INFO: `/v3/domains` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:08.524984 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca97558-4482-4abd-9efd-a563bb3c8b69 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:52:08.525291 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca97558-4482-4abd-9efd-a563bb3c8b69 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:08.526905 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca97558-4482-4abd-9efd-a563bb3c8b69 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:08.539562 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1405/2769] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jun 7 14:52:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:52:08.549112 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e57947-b6df-4a4c-8074-ae8a97b9056d None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:08.558083 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e57947-b6df-4a4c-8074-ae8a97b9056d None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:08.562718 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5886b7-caec-46db-9be9-42b297b72440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:08.563418 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5886b7-caec-46db-9be9-42b297b72440 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:08.563726 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5886b7-caec-46db-9be9-42b297b72440 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:08.563977 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5886b7-caec-46db-9be9-42b297b72440 admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:08.564716 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5886b7-caec-46db-9be9-42b297b72440 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:08.566588 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5886b7-caec-46db-9be9-42b297b72440 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:08.664065 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1365/2770] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:52:08 2022] POST /identity/v3/projects => generated 381 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:52:08.668459 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca97558-4482-4abd-9efd-a563bb3c8b69 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:08.679678 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca97558-4482-4abd-9efd-a563bb3c8b69 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:08.710112 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:52:08.737319 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f14a95-1b6c-4200-955c-730a3a5c2cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:08.738307 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f14a95-1b6c-4200-955c-730a3a5c2cbf admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:08.738689 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f14a95-1b6c-4200-955c-730a3a5c2cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:08.738975 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f14a95-1b6c-4200-955c-730a3a5c2cbf admin admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:08.739678 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f14a95-1b6c-4200-955c-730a3a5c2cbf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:08.741685 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f14a95-1b6c-4200-955c-730a3a5c2cbf admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:08.845602 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1406/2771] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:52:08 2022] POST /identity/v3/users => generated 291 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:52:08.850006 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5886b7-caec-46db-9be9-42b297b72440 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:08.861286 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5886b7-caec-46db-9be9-42b297b72440 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:08.866012 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2ec9fc-bc66-40e2-a13d-a40a2982489d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:08.866685 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2ec9fc-bc66-40e2-a13d-a40a2982489d admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:08.866977 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2ec9fc-bc66-40e2-a13d-a40a2982489d admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:08.867292 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2ec9fc-bc66-40e2-a13d-a40a2982489d admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:08.868052 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2ec9fc-bc66-40e2-a13d-a40a2982489d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:08.869718 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2ec9fc-bc66-40e2-a13d-a40a2982489d admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:08.880459 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1366/2772] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:52:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:52:08.884476 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f14a95-1b6c-4200-955c-730a3a5c2cbf admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:08.895611 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f14a95-1b6c-4200-955c-730a3a5c2cbf admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:08.900637 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f334b66-31fa-4c5a-9b58-030d7851dfb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:08.901355 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f334b66-31fa-4c5a-9b58-030d7851dfb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:08.901590 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f334b66-31fa-4c5a-9b58-030d7851dfb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:08.901872 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f334b66-31fa-4c5a-9b58-030d7851dfb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:08.902870 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f334b66-31fa-4c5a-9b58-030d7851dfb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:08.904486 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f334b66-31fa-4c5a-9b58-030d7851dfb1 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:08.914751 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1407/2773] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:52:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:52:08.918899 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2ec9fc-bc66-40e2-a13d-a40a2982489d admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:08.929034 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2ec9fc-bc66-40e2-a13d-a40a2982489d admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:08.933714 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69982b9c-3e24-4228-a941-88b7c7fc9797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:08.934722 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69982b9c-3e24-4228-a941-88b7c7fc9797 admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:08.935023 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69982b9c-3e24-4228-a941-88b7c7fc9797 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:08.935157 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69982b9c-3e24-4228-a941-88b7c7fc9797 admin admin] PATH_INFO: `/v3/projects/770e67d33e6146ee9a41005a3a930a2e/users/1570dcaa62164ff7b48d92c8c9885c81/roles/3771341b3be641139e752aab690397a4` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:08.950480 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69982b9c-3e24-4228-a941-88b7c7fc9797 admin admin] RBAC: Authorizing `identity:create_grant(project_id=770e67d33e6146ee9a41005a3a930a2e, user_id=1570dcaa62164ff7b48d92c8c9885c81, role_id=3771341b3be641139e752aab690397a4)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:08.953789 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69982b9c-3e24-4228-a941-88b7c7fc9797 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:09.045877 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1367/2774] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:52:08 2022] PUT /identity/v3/projects/770e67d33e6146ee9a41005a3a930a2e/users/1570dcaa62164ff7b48d92c8c9885c81/roles/3771341b3be641139e752aab690397a4 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:52:09.056813 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8547493-73e7-41b5-95bd-5c12ca55289e None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:09.057128 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8547493-73e7-41b5-95bd-5c12ca55289e None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:09.057409 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8547493-73e7-41b5-95bd-5c12ca55289e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:09.084914 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-b8547493-73e7-41b5-95bd-5c12ca55289e None None] MFA Rules not processed for user `1570dcaa62164ff7b48d92c8c9885c81`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:52:09.086126 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-b8547493-73e7-41b5-95bd-5c12ca55289e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:52:09.148649 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1408/2775] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:52:09 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:52:09.153932 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69982b9c-3e24-4228-a941-88b7c7fc9797 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:09.215991 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69982b9c-3e24-4228-a941-88b7c7fc9797 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:09.221253 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c841f3-15a0-4c05-83c7-86e533a1b564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:09.221904 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c841f3-15a0-4c05-83c7-86e533a1b564 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:09.222192 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c841f3-15a0-4c05-83c7-86e533a1b564 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:09.222465 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c841f3-15a0-4c05-83c7-86e533a1b564 admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:09.223150 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c841f3-15a0-4c05-83c7-86e533a1b564 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:09.225085 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c841f3-15a0-4c05-83c7-86e533a1b564 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:09.360737 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1368/2776] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:52:09 2022] POST /identity/v3/projects => generated 381 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:52:09.366489 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8547493-73e7-41b5-95bd-5c12ca55289e None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:09.379920 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8547493-73e7-41b5-95bd-5c12ca55289e None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:09.436323 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7b75b6-8c64-46d0-ae3e-c436576b2762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:09.437020 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7b75b6-8c64-46d0-ae3e-c436576b2762 admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:09.437301 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7b75b6-8c64-46d0-ae3e-c436576b2762 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:09.438084 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7b75b6-8c64-46d0-ae3e-c436576b2762 admin admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:09.438837 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7b75b6-8c64-46d0-ae3e-c436576b2762 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:09.440759 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7b75b6-8c64-46d0-ae3e-c436576b2762 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:09.542820 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1409/2777] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:52:09 2022] POST /identity/v3/users => generated 297 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:52:09.546900 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c841f3-15a0-4c05-83c7-86e533a1b564 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:09.557828 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c841f3-15a0-4c05-83c7-86e533a1b564 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:09.563185 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4cb0d5-1b64-48d9-99d8-a3bfbee40148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:09.564045 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4cb0d5-1b64-48d9-99d8-a3bfbee40148 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:09.564318 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4cb0d5-1b64-48d9-99d8-a3bfbee40148 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:09.564633 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4cb0d5-1b64-48d9-99d8-a3bfbee40148 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:09.565356 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4cb0d5-1b64-48d9-99d8-a3bfbee40148 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:09.566942 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4cb0d5-1b64-48d9-99d8-a3bfbee40148 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:09.577552 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1369/2778] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:52:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:52:09.582356 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7b75b6-8c64-46d0-ae3e-c436576b2762 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:09.592314 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7b75b6-8c64-46d0-ae3e-c436576b2762 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:09.596897 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c0b436-f154-4c22-bf78-c72b99667723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:09.597771 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c0b436-f154-4c22-bf78-c72b99667723 admin admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:09.598055 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c0b436-f154-4c22-bf78-c72b99667723 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:09.598332 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c0b436-f154-4c22-bf78-c72b99667723 admin admin] PATH_INFO: `/v3/projects/544364d7838a48559b3e23621adb7fc1/users/ef0a855851324756bc61b445f6420bfe/roles/9baff2b18f2649f7b757ff2e1f63c853` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:09.613007 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c0b436-f154-4c22-bf78-c72b99667723 admin admin] RBAC: Authorizing `identity:create_grant(project_id=544364d7838a48559b3e23621adb7fc1, user_id=ef0a855851324756bc61b445f6420bfe, role_id=9baff2b18f2649f7b757ff2e1f63c853)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:09.615722 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c0b436-f154-4c22-bf78-c72b99667723 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:09.686008 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1410/2779] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:52:09 2022] PUT /identity/v3/projects/544364d7838a48559b3e23621adb7fc1/users/ef0a855851324756bc61b445f6420bfe/roles/9baff2b18f2649f7b757ff2e1f63c853 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:52:09.690601 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4cb0d5-1b64-48d9-99d8-a3bfbee40148 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:09.700678 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4cb0d5-1b64-48d9-99d8-a3bfbee40148 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:09.759745 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dae487c-51b9-4e63-bcaf-c642151c2fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:09.760348 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dae487c-51b9-4e63-bcaf-c642151c2fd8 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:09.760687 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dae487c-51b9-4e63-bcaf-c642151c2fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:09.760968 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dae487c-51b9-4e63-bcaf-c642151c2fd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:09.761710 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dae487c-51b9-4e63-bcaf-c642151c2fd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:09.763347 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dae487c-51b9-4e63-bcaf-c642151c2fd8 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:09.765601 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:52:09.773947 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1370/2780] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:52:09 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:52:09.778464 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c0b436-f154-4c22-bf78-c72b99667723 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:09.789103 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c0b436-f154-4c22-bf78-c72b99667723 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:09.793705 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6cdf07-9220-4f66-9d90-8847981bb3b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:09.794668 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6cdf07-9220-4f66-9d90-8847981bb3b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:09.794953 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6cdf07-9220-4f66-9d90-8847981bb3b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:09.795227 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6cdf07-9220-4f66-9d90-8847981bb3b9 admin admin] PATH_INFO: `/v3/projects/544364d7838a48559b3e23621adb7fc1/users/ef0a855851324756bc61b445f6420bfe/roles/9baff2b18f2649f7b757ff2e1f63c853` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:09.797817 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6cdf07-9220-4f66-9d90-8847981bb3b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=544364d7838a48559b3e23621adb7fc1, user_id=ef0a855851324756bc61b445f6420bfe, role_id=9baff2b18f2649f7b757ff2e1f63c853)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:09.800566 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6cdf07-9220-4f66-9d90-8847981bb3b9 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:09.803133 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:52:09.811965 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1411/2781] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:52:09 2022] PUT /identity/v3/projects/544364d7838a48559b3e23621adb7fc1/users/ef0a855851324756bc61b445f6420bfe/roles/9baff2b18f2649f7b757ff2e1f63c853 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:52:09.823039 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251b1611-30cd-4e10-90d0-4884bbbd415d None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:09.823449 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251b1611-30cd-4e10-90d0-4884bbbd415d None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:09.823836 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251b1611-30cd-4e10-90d0-4884bbbd415d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:09.849992 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-251b1611-30cd-4e10-90d0-4884bbbd415d None None] MFA Rules not processed for user `ef0a855851324756bc61b445f6420bfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:52:09.851194 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-251b1611-30cd-4e10-90d0-4884bbbd415d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:52:09.919714 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1371/2782] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:52:09 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:52:09.931080 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6cdf07-9220-4f66-9d90-8847981bb3b9 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:09.932801 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-8f6cdf07-9220-4f66-9d90-8847981bb3b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:52:09.991271 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6cdf07-9220-4f66-9d90-8847981bb3b9 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:09.995107 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78c5fd1-96a9-48af-ab48-4b04736bfe32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d81f597cb44c56a966537089bcce4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:09.995754 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78c5fd1-96a9-48af-ab48-4b04736bfe32 service cinder] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:09.996045 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78c5fd1-96a9-48af-ab48-4b04736bfe32 service cinder] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:09.996326 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78c5fd1-96a9-48af-ab48-4b04736bfe32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:10.006872 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78c5fd1-96a9-48af-ab48-4b04736bfe32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:10.008306 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78c5fd1-96a9-48af-ab48-4b04736bfe32 service cinder] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:10.018151 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1412/2783] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:52:09 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:52:10.115805 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251b1611-30cd-4e10-90d0-4884bbbd415d None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:10.126077 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251b1611-30cd-4e10-90d0-4884bbbd415d None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:10.129771 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041c62c9-9bee-4a77-bd9d-79b695e21ddb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d81f597cb44c56a966537089bcce4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:10.130417 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041c62c9-9bee-4a77-bd9d-79b695e21ddb service cinder] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:10.130703 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041c62c9-9bee-4a77-bd9d-79b695e21ddb service cinder] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:10.130982 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041c62c9-9bee-4a77-bd9d-79b695e21ddb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:10.188539 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041c62c9-9bee-4a77-bd9d-79b695e21ddb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:10.189970 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041c62c9-9bee-4a77-bd9d-79b695e21ddb service cinder] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:10.198837 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1372/2784] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:52:10 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:52:17.538536 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78c5fd1-96a9-48af-ab48-4b04736bfe32 service cinder] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:17.603186 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78c5fd1-96a9-48af-ab48-4b04736bfe32 service cinder] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:17.607858 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505dd865-ff7a-40e4-90d7-9ca38dfe45e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:17.608768 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505dd865-ff7a-40e4-90d7-9ca38dfe45e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:17.609055 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505dd865-ff7a-40e4-90d7-9ca38dfe45e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:17.609333 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505dd865-ff7a-40e4-90d7-9ca38dfe45e7 admin admin] PATH_INFO: `/v3/users/1570dcaa62164ff7b48d92c8c9885c81` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:17.610622 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505dd865-ff7a-40e4-90d7-9ca38dfe45e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1570dcaa62164ff7b48d92c8c9885c81)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:17.612668 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505dd865-ff7a-40e4-90d7-9ca38dfe45e7 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:17.676260 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041c62c9-9bee-4a77-bd9d-79b695e21ddb service cinder] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:17.677754 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-041c62c9-9bee-4a77-bd9d-79b695e21ddb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:52:17.691923 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041c62c9-9bee-4a77-bd9d-79b695e21ddb service cinder] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:17.696466 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02840f4f-01fd-4d0e-b0b5-29450cf23d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:17.697401 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02840f4f-01fd-4d0e-b0b5-29450cf23d82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:17.697689 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02840f4f-01fd-4d0e-b0b5-29450cf23d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:17.697994 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02840f4f-01fd-4d0e-b0b5-29450cf23d82 admin admin] PATH_INFO: `/v3/users/bd5cf2faa24c4e14a86081cd5b004a86` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:17.699181 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02840f4f-01fd-4d0e-b0b5-29450cf23d82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd5cf2faa24c4e14a86081cd5b004a86)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:17.701181 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02840f4f-01fd-4d0e-b0b5-29450cf23d82 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:17.848679 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-505dd865-ff7a-40e4-90d7-9ca38dfe45e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1570dcaa62164ff7b48d92c8c9885c81'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:17.849297 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-505dd865-ff7a-40e4-90d7-9ca38dfe45e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1570dcaa62164ff7b48d92c8c9885c81'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:17.849910 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-505dd865-ff7a-40e4-90d7-9ca38dfe45e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1570dcaa62164ff7b48d92c8c9885c81'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:17.856861 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-505dd865-ff7a-40e4-90d7-9ca38dfe45e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1570dcaa62164ff7b48d92c8c9885c81'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:17.868924 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-505dd865-ff7a-40e4-90d7-9ca38dfe45e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1570dcaa62164ff7b48d92c8c9885c81'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:17.922787 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-02840f4f-01fd-4d0e-b0b5-29450cf23d82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd5cf2faa24c4e14a86081cd5b004a86'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:17.923432 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-02840f4f-01fd-4d0e-b0b5-29450cf23d82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd5cf2faa24c4e14a86081cd5b004a86'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:17.924051 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-02840f4f-01fd-4d0e-b0b5-29450cf23d82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd5cf2faa24c4e14a86081cd5b004a86'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:17.930860 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-02840f4f-01fd-4d0e-b0b5-29450cf23d82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd5cf2faa24c4e14a86081cd5b004a86'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:17.944102 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-02840f4f-01fd-4d0e-b0b5-29450cf23d82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd5cf2faa24c4e14a86081cd5b004a86'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:17.946811 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:52:17.950379 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1413/2785] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:52:17 2022] DELETE /identity/v3/users/1570dcaa62164ff7b48d92c8c9885c81 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:52:17.955699 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf276bc-f690-4ba9-a08d-b920f7117958 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:17.956320 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf276bc-f690-4ba9-a08d-b920f7117958 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:17.956631 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf276bc-f690-4ba9-a08d-b920f7117958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:17.975025 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-0bf276bc-f690-4ba9-a08d-b920f7117958 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:52:17.976231 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-0bf276bc-f690-4ba9-a08d-b920f7117958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:52:17.986572 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:52:18.035918 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1414/2786] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:52:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:52:18.046238 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf276bc-f690-4ba9-a08d-b920f7117958 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:18.047730 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-0bf276bc-f690-4ba9-a08d-b920f7117958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:52:18.077527 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1373/2787] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:52:17 2022] DELETE /identity/v3/users/bd5cf2faa24c4e14a86081cd5b004a86 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:52:18.082436 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a8aa51-fdab-4fe4-a572-6e07267ba65e None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:18.082737 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a8aa51-fdab-4fe4-a572-6e07267ba65e None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:18.083023 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a8aa51-fdab-4fe4-a572-6e07267ba65e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:18.101693 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-99a8aa51-fdab-4fe4-a572-6e07267ba65e None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:52:18.102811 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-99a8aa51-fdab-4fe4-a572-6e07267ba65e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:52:18.115362 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf276bc-f690-4ba9-a08d-b920f7117958 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:18.167806 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7ec2dc-35a7-4521-b6d9-e9e35fcb12a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:18.168574 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7ec2dc-35a7-4521-b6d9-e9e35fcb12a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:18.168866 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7ec2dc-35a7-4521-b6d9-e9e35fcb12a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:18.169154 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7ec2dc-35a7-4521-b6d9-e9e35fcb12a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:18.174768 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1374/2788] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:52:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:52:18.184177 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a8aa51-fdab-4fe4-a572-6e07267ba65e None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:18.185592 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-99a8aa51-fdab-4fe4-a572-6e07267ba65e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:52:18.189666 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7ec2dc-35a7-4521-b6d9-e9e35fcb12a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:18.189666 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7ec2dc-35a7-4521-b6d9-e9e35fcb12a4 service neutron] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:18.198512 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a8aa51-fdab-4fe4-a572-6e07267ba65e None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:18.199943 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1415/2789] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:52:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 14:52:18.203574 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58c8463-3efe-42b9-a49c-5e1a8deb4c2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:18.204236 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58c8463-3efe-42b9-a49c-5e1a8deb4c2e service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:18.204560 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58c8463-3efe-42b9-a49c-5e1a8deb4c2e service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:18.204848 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58c8463-3efe-42b9-a49c-5e1a8deb4c2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:18.215104 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58c8463-3efe-42b9-a49c-5e1a8deb4c2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:18.216567 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58c8463-3efe-42b9-a49c-5e1a8deb4c2e service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:18.227509 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1375/2790] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:52:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 14:52:18.751641 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7ec2dc-35a7-4521-b6d9-e9e35fcb12a4 service neutron] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:18.755855 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-dd7ec2dc-35a7-4521-b6d9-e9e35fcb12a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:52:18.756024 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58c8463-3efe-42b9-a49c-5e1a8deb4c2e service neutron] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:18.758195 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-b58c8463-3efe-42b9-a49c-5e1a8deb4c2e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:52:18.769358 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7ec2dc-35a7-4521-b6d9-e9e35fcb12a4 service neutron] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:18.774940 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58c8463-3efe-42b9-a49c-5e1a8deb4c2e service neutron] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:18.775113 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4114f8a9-466f-4147-aaee-35f9f7aa001e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:18.775935 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4114f8a9-466f-4147-aaee-35f9f7aa001e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:18.776226 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4114f8a9-466f-4147-aaee-35f9f7aa001e admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:18.776544 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4114f8a9-466f-4147-aaee-35f9f7aa001e admin admin] PATH_INFO: `/v3/projects/3a03a2b8deed46b9a96ab27eed222051` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:18.778110 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4114f8a9-466f-4147-aaee-35f9f7aa001e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a03a2b8deed46b9a96ab27eed222051)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:18.780107 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4114f8a9-466f-4147-aaee-35f9f7aa001e admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:18.801459 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaa6e7f-0db8-49f1-a4cf-5b116fb168bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:18.802533 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaa6e7f-0db8-49f1-a4cf-5b116fb168bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:18.802823 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaa6e7f-0db8-49f1-a4cf-5b116fb168bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:18.803108 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaa6e7f-0db8-49f1-a4cf-5b116fb168bc admin admin] PATH_INFO: `/v3/projects/770e67d33e6146ee9a41005a3a930a2e` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:18.804472 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaa6e7f-0db8-49f1-a4cf-5b116fb168bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=770e67d33e6146ee9a41005a3a930a2e)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:18.806419 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaa6e7f-0db8-49f1-a4cf-5b116fb168bc admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:18.893174 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-4114f8a9-466f-4147-aaee-35f9f7aa001e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a03a2b8deed46b9a96ab27eed222051', 'request_id': 'req-4114f8a9-466f-4147-aaee-35f9f7aa001e'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:18.961846 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-fdaa6e7f-0db8-49f1-a4cf-5b116fb168bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '770e67d33e6146ee9a41005a3a930a2e', 'request_id': 'req-fdaa6e7f-0db8-49f1-a4cf-5b116fb168bc'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:18.963749 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-4114f8a9-466f-4147-aaee-35f9f7aa001e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a03a2b8deed46b9a96ab27eed222051', 'request_id': 'req-4114f8a9-466f-4147-aaee-35f9f7aa001e'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:18.973864 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-4114f8a9-466f-4147-aaee-35f9f7aa001e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a03a2b8deed46b9a96ab27eed222051', 'request_id': 'req-4114f8a9-466f-4147-aaee-35f9f7aa001e'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:18.978202 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:52:18.982822 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-4114f8a9-466f-4147-aaee-35f9f7aa001e admin admin] The token cache is being invalidate because project 3a03a2b8deed46b9a96ab27eed222051 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:52:18.983107 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-4114f8a9-466f-4147-aaee-35f9f7aa001e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:18.983750 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-4114f8a9-466f-4147-aaee-35f9f7aa001e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:18.985413 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1416/2791] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:52:18 2022] DELETE /identity/v3/projects/3a03a2b8deed46b9a96ab27eed222051 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:52:19.061189 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-fdaa6e7f-0db8-49f1-a4cf-5b116fb168bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '770e67d33e6146ee9a41005a3a930a2e', 'request_id': 'req-fdaa6e7f-0db8-49f1-a4cf-5b116fb168bc'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:19.067996 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-fdaa6e7f-0db8-49f1-a4cf-5b116fb168bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '770e67d33e6146ee9a41005a3a930a2e', 'request_id': 'req-fdaa6e7f-0db8-49f1-a4cf-5b116fb168bc'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:19.080848 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-fdaa6e7f-0db8-49f1-a4cf-5b116fb168bc admin admin] The token cache is being invalidate because project 770e67d33e6146ee9a41005a3a930a2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:52:19.081142 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-fdaa6e7f-0db8-49f1-a4cf-5b116fb168bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:19.081824 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-fdaa6e7f-0db8-49f1-a4cf-5b116fb168bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:19.083444 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1376/2792] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:52:18 2022] DELETE /identity/v3/projects/770e67d33e6146ee9a41005a3a930a2e => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:52:19.087756 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4114f8a9-466f-4147-aaee-35f9f7aa001e admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:19.089308 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-4114f8a9-466f-4147-aaee-35f9f7aa001e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:52:19.161221 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4114f8a9-466f-4147-aaee-35f9f7aa001e admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:19.226488 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0badda52-8b2d-44f3-b996-c2f931ac8d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:19.227381 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0badda52-8b2d-44f3-b996-c2f931ac8d42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:19.227663 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0badda52-8b2d-44f3-b996-c2f931ac8d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:19.227937 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0badda52-8b2d-44f3-b996-c2f931ac8d42 admin admin] PATH_INFO: `/v3/users/ef0a855851324756bc61b445f6420bfe` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:19.229246 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0badda52-8b2d-44f3-b996-c2f931ac8d42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef0a855851324756bc61b445f6420bfe)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:19.231213 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0badda52-8b2d-44f3-b996-c2f931ac8d42 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:19.447394 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-0badda52-8b2d-44f3-b996-c2f931ac8d42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef0a855851324756bc61b445f6420bfe'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:19.448038 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-0badda52-8b2d-44f3-b996-c2f931ac8d42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef0a855851324756bc61b445f6420bfe'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:19.448690 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-0badda52-8b2d-44f3-b996-c2f931ac8d42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef0a855851324756bc61b445f6420bfe'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:19.456100 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-0badda52-8b2d-44f3-b996-c2f931ac8d42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef0a855851324756bc61b445f6420bfe'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:19.469956 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-0badda52-8b2d-44f3-b996-c2f931ac8d42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef0a855851324756bc61b445f6420bfe'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:19.560602 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1417/2793] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:52:19 2022] DELETE /identity/v3/users/ef0a855851324756bc61b445f6420bfe => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:52:19.933287 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaa6e7f-0db8-49f1-a4cf-5b116fb168bc admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:19.934854 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-fdaa6e7f-0db8-49f1-a4cf-5b116fb168bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:52:19.953617 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaa6e7f-0db8-49f1-a4cf-5b116fb168bc admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:20.008399 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a17c57-2d19-4008-ba52-28af4e2366a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:20.009258 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a17c57-2d19-4008-ba52-28af4e2366a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:20.009542 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a17c57-2d19-4008-ba52-28af4e2366a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:20.009820 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a17c57-2d19-4008-ba52-28af4e2366a7 admin admin] PATH_INFO: `/v3/projects/544364d7838a48559b3e23621adb7fc1` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:20.010999 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a17c57-2d19-4008-ba52-28af4e2366a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=544364d7838a48559b3e23621adb7fc1)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:20.012924 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a17c57-2d19-4008-ba52-28af4e2366a7 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:20.019613 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:52:20.137196 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-15a17c57-2d19-4008-ba52-28af4e2366a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '544364d7838a48559b3e23621adb7fc1', 'request_id': 'req-15a17c57-2d19-4008-ba52-28af4e2366a7'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:20.227050 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-15a17c57-2d19-4008-ba52-28af4e2366a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '544364d7838a48559b3e23621adb7fc1', 'request_id': 'req-15a17c57-2d19-4008-ba52-28af4e2366a7'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:20.233118 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-15a17c57-2d19-4008-ba52-28af4e2366a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '544364d7838a48559b3e23621adb7fc1', 'request_id': 'req-15a17c57-2d19-4008-ba52-28af4e2366a7'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:20.244648 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-15a17c57-2d19-4008-ba52-28af4e2366a7 admin admin] The token cache is being invalidate because project 544364d7838a48559b3e23621adb7fc1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:52:20.244936 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-15a17c57-2d19-4008-ba52-28af4e2366a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:20.245575 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-15a17c57-2d19-4008-ba52-28af4e2366a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:20.247221 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1377/2794] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:52:19 2022] DELETE /identity/v3/projects/544364d7838a48559b3e23621adb7fc1 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:52:20.252689 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c73bbd4-3d22-49cd-9a87-2c8c5bc65ba1 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:20.253189 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c73bbd4-3d22-49cd-9a87-2c8c5bc65ba1 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:20.253287 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c73bbd4-3d22-49cd-9a87-2c8c5bc65ba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:20.271663 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-0c73bbd4-3d22-49cd-9a87-2c8c5bc65ba1 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:52:20.272898 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-0c73bbd4-3d22-49cd-9a87-2c8c5bc65ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:52:20.292123 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:52:20.340274 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1418/2795] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:52:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:52:20.342312 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb188c4-2777-49ff-8544-e48c01487274 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:20.343159 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb188c4-2777-49ff-8544-e48c01487274 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:20.343159 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb188c4-2777-49ff-8544-e48c01487274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:20.365268 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-afb188c4-2777-49ff-8544-e48c01487274 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:52:20.366489 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-afb188c4-2777-49ff-8544-e48c01487274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:52:20.375657 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1378/2796] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:52:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:52:20.380459 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c73bbd4-3d22-49cd-9a87-2c8c5bc65ba1 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:20.389874 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c73bbd4-3d22-49cd-9a87-2c8c5bc65ba1 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:20.406681 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f4eb8b-4fce-4956-9d69-e1ce93d36034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:20.406681 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f4eb8b-4fce-4956-9d69-e1ce93d36034 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:20.407104 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f4eb8b-4fce-4956-9d69-e1ce93d36034 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:20.407399 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f4eb8b-4fce-4956-9d69-e1ce93d36034 admin admin] PATH_INFO: `/v3/domains` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:20.408164 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f4eb8b-4fce-4956-9d69-e1ce93d36034 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:52:20.408510 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f4eb8b-4fce-4956-9d69-e1ce93d36034 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:20.410209 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f4eb8b-4fce-4956-9d69-e1ce93d36034 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:20.424298 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1419/2797] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jun 7 14:52:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:52:20.428683 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb188c4-2777-49ff-8544-e48c01487274 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:20.438761 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb188c4-2777-49ff-8544-e48c01487274 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:20.445677 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9832f7e7-604a-432f-9eca-4b8528d7c516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:20.446348 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9832f7e7-604a-432f-9eca-4b8528d7c516 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:20.446631 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9832f7e7-604a-432f-9eca-4b8528d7c516 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:20.446905 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9832f7e7-604a-432f-9eca-4b8528d7c516 admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:20.447598 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9832f7e7-604a-432f-9eca-4b8528d7c516 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:20.449627 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9832f7e7-604a-432f-9eca-4b8528d7c516 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:20.539799 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1379/2798] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:52:20 2022] POST /identity/v3/projects => generated 385 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:52:20.544061 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f4eb8b-4fce-4956-9d69-e1ce93d36034 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:20.560455 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f4eb8b-4fce-4956-9d69-e1ce93d36034 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:20.615009 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86fe98a-6de6-4e05-871d-57f8fc412bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:20.615702 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86fe98a-6de6-4e05-871d-57f8fc412bd8 admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:20.615992 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86fe98a-6de6-4e05-871d-57f8fc412bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:20.616294 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86fe98a-6de6-4e05-871d-57f8fc412bd8 admin admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:20.617020 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86fe98a-6de6-4e05-871d-57f8fc412bd8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:20.618926 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86fe98a-6de6-4e05-871d-57f8fc412bd8 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:20.710685 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1420/2799] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:52:20 2022] POST /identity/v3/users => generated 293 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:52:20.718220 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9832f7e7-604a-432f-9eca-4b8528d7c516 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:20.735951 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9832f7e7-604a-432f-9eca-4b8528d7c516 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:20.741211 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d461377-d15e-4de9-850d-98f94ec8b3b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:20.741877 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d461377-d15e-4de9-850d-98f94ec8b3b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:20.742165 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d461377-d15e-4de9-850d-98f94ec8b3b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:20.742447 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d461377-d15e-4de9-850d-98f94ec8b3b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:20.743186 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d461377-d15e-4de9-850d-98f94ec8b3b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:20.744877 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d461377-d15e-4de9-850d-98f94ec8b3b2 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:20.755813 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1380/2800] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:52:20 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:52:20.760037 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86fe98a-6de6-4e05-871d-57f8fc412bd8 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:20.772999 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86fe98a-6de6-4e05-871d-57f8fc412bd8 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:20.778068 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c0962f-2f1d-418d-a300-9bf29d89f121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:20.778732 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c0962f-2f1d-418d-a300-9bf29d89f121 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:20.779021 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c0962f-2f1d-418d-a300-9bf29d89f121 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:20.779365 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c0962f-2f1d-418d-a300-9bf29d89f121 admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:20.780099 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c0962f-2f1d-418d-a300-9bf29d89f121 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:20.782286 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c0962f-2f1d-418d-a300-9bf29d89f121 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:20.792623 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1421/2801] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:52:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 14:52:20.797415 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d461377-d15e-4de9-850d-98f94ec8b3b2 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:20.807545 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d461377-d15e-4de9-850d-98f94ec8b3b2 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:20.812341 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8379b2-c7c6-4930-ac20-13bb09170aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:20.813256 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8379b2-c7c6-4930-ac20-13bb09170aaf admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:20.813580 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8379b2-c7c6-4930-ac20-13bb09170aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:20.813860 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8379b2-c7c6-4930-ac20-13bb09170aaf admin admin] PATH_INFO: `/v3/projects/c25d90665fb548d79ea3902ec50dc236/users/412a237e9c4b4c15bfb688a07a57d114/roles/3771341b3be641139e752aab690397a4` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:20.828762 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8379b2-c7c6-4930-ac20-13bb09170aaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=c25d90665fb548d79ea3902ec50dc236, user_id=412a237e9c4b4c15bfb688a07a57d114, role_id=3771341b3be641139e752aab690397a4)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:20.831428 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8379b2-c7c6-4930-ac20-13bb09170aaf admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:20.935162 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1381/2802] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:52:20 2022] PUT /identity/v3/projects/c25d90665fb548d79ea3902ec50dc236/users/412a237e9c4b4c15bfb688a07a57d114/roles/3771341b3be641139e752aab690397a4 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:52:20.945907 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afc194b-346e-41e2-a4f6-5b6b6060ea46 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:20.946201 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afc194b-346e-41e2-a4f6-5b6b6060ea46 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:20.948629 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afc194b-346e-41e2-a4f6-5b6b6060ea46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:20.975276 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-2afc194b-346e-41e2-a4f6-5b6b6060ea46 None None] MFA Rules not processed for user `412a237e9c4b4c15bfb688a07a57d114`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:52:20.976552 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-2afc194b-346e-41e2-a4f6-5b6b6060ea46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:52:21.042411 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1422/2803] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:52:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:52:21.051010 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8379b2-c7c6-4930-ac20-13bb09170aaf admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:21.112459 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8379b2-c7c6-4930-ac20-13bb09170aaf admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:21.117105 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bab54b-0224-4236-a1d4-532640798249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:21.117760 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bab54b-0224-4236-a1d4-532640798249 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:21.118048 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bab54b-0224-4236-a1d4-532640798249 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:21.118360 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bab54b-0224-4236-a1d4-532640798249 admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:21.119045 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bab54b-0224-4236-a1d4-532640798249 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:21.121011 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bab54b-0224-4236-a1d4-532640798249 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:21.215973 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1382/2804] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:52:21 2022] POST /identity/v3/projects => generated 387 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:52:21.220066 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afc194b-346e-41e2-a4f6-5b6b6060ea46 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:21.230658 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afc194b-346e-41e2-a4f6-5b6b6060ea46 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:21.247294 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:52:21.287277 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1c2b6d-827c-4bc4-a527-52db4f181132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:21.287960 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1c2b6d-827c-4bc4-a527-52db4f181132 admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:21.288265 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1c2b6d-827c-4bc4-a527-52db4f181132 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:21.288865 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1c2b6d-827c-4bc4-a527-52db4f181132 admin admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:21.289508 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1c2b6d-827c-4bc4-a527-52db4f181132 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:21.291422 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1c2b6d-827c-4bc4-a527-52db4f181132 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:21.383871 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1423/2805] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:52:21 2022] POST /identity/v3/users => generated 300 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:52:21.387818 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bab54b-0224-4236-a1d4-532640798249 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:21.398203 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bab54b-0224-4236-a1d4-532640798249 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:21.403048 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5fe222-24ff-4815-bc4a-94637e1aabbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:21.403747 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5fe222-24ff-4815-bc4a-94637e1aabbd admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:21.404029 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5fe222-24ff-4815-bc4a-94637e1aabbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:21.404305 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5fe222-24ff-4815-bc4a-94637e1aabbd admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:21.405057 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5fe222-24ff-4815-bc4a-94637e1aabbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:21.407170 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5fe222-24ff-4815-bc4a-94637e1aabbd admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:21.417679 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1383/2806] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:52:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:52:21.422496 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1c2b6d-827c-4bc4-a527-52db4f181132 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:21.432559 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1c2b6d-827c-4bc4-a527-52db4f181132 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:21.437149 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a29322b-5a35-46be-b2e5-d6dabaf2e9bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:21.438022 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a29322b-5a35-46be-b2e5-d6dabaf2e9bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:21.438305 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a29322b-5a35-46be-b2e5-d6dabaf2e9bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:21.438583 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a29322b-5a35-46be-b2e5-d6dabaf2e9bc admin admin] PATH_INFO: `/v3/projects/b6084bad35434820ab0ecefe99b95e4e/users/178b954bafb54fbf95dd39e738cb3b79/roles/9baff2b18f2649f7b757ff2e1f63c853` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:21.453007 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a29322b-5a35-46be-b2e5-d6dabaf2e9bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6084bad35434820ab0ecefe99b95e4e, user_id=178b954bafb54fbf95dd39e738cb3b79, role_id=9baff2b18f2649f7b757ff2e1f63c853)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:21.455644 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a29322b-5a35-46be-b2e5-d6dabaf2e9bc admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:21.543619 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1424/2807] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:52:21 2022] PUT /identity/v3/projects/b6084bad35434820ab0ecefe99b95e4e/users/178b954bafb54fbf95dd39e738cb3b79/roles/9baff2b18f2649f7b757ff2e1f63c853 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:52:21.551860 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5fe222-24ff-4815-bc4a-94637e1aabbd admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:21.554474 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:52:21.559374 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5fe222-24ff-4815-bc4a-94637e1aabbd admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:21.616419 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08c741a-886e-4d3d-b183-9d53116eb97a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:21.617100 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08c741a-886e-4d3d-b183-9d53116eb97a admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:21.617389 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08c741a-886e-4d3d-b183-9d53116eb97a admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:21.617672 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08c741a-886e-4d3d-b183-9d53116eb97a admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:21.618399 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08c741a-886e-4d3d-b183-9d53116eb97a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:21.620008 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08c741a-886e-4d3d-b183-9d53116eb97a admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:21.629874 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1384/2808] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:52:21 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:52:21.634253 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a29322b-5a35-46be-b2e5-d6dabaf2e9bc admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:21.644746 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a29322b-5a35-46be-b2e5-d6dabaf2e9bc admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:21.649555 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61db56a-6819-4870-bf49-f2be5beaaf6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:21.650556 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61db56a-6819-4870-bf49-f2be5beaaf6e admin admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:21.650911 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61db56a-6819-4870-bf49-f2be5beaaf6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:21.651289 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61db56a-6819-4870-bf49-f2be5beaaf6e admin admin] PATH_INFO: `/v3/projects/b6084bad35434820ab0ecefe99b95e4e/users/178b954bafb54fbf95dd39e738cb3b79/roles/9baff2b18f2649f7b757ff2e1f63c853` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:21.653726 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61db56a-6819-4870-bf49-f2be5beaaf6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6084bad35434820ab0ecefe99b95e4e, user_id=178b954bafb54fbf95dd39e738cb3b79, role_id=9baff2b18f2649f7b757ff2e1f63c853)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:21.657105 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61db56a-6819-4870-bf49-f2be5beaaf6e admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:21.669035 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1425/2809] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:52:21 2022] PUT /identity/v3/projects/b6084bad35434820ab0ecefe99b95e4e/users/178b954bafb54fbf95dd39e738cb3b79/roles/9baff2b18f2649f7b757ff2e1f63c853 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:52:21.681094 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4c58e4-970d-47b3-81cf-c1b63936ca90 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:21.681620 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4c58e4-970d-47b3-81cf-c1b63936ca90 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:21.682000 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4c58e4-970d-47b3-81cf-c1b63936ca90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:21.709297 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-1d4c58e4-970d-47b3-81cf-c1b63936ca90 None None] MFA Rules not processed for user `178b954bafb54fbf95dd39e738cb3b79`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:52:21.710494 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-1d4c58e4-970d-47b3-81cf-c1b63936ca90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:52:21.794535 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1385/2810] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:52:21 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:52:21.805648 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61db56a-6819-4870-bf49-f2be5beaaf6e admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:21.807388 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-a61db56a-6819-4870-bf49-f2be5beaaf6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:52:21.861696 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4c58e4-970d-47b3-81cf-c1b63936ca90 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:21.863950 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-1d4c58e4-970d-47b3-81cf-c1b63936ca90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:52:21.870223 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61db56a-6819-4870-bf49-f2be5beaaf6e admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:21.873972 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cfe756-6096-429f-b8af-c4f80db91eee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d81f597cb44c56a966537089bcce4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:21.874795 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cfe756-6096-429f-b8af-c4f80db91eee service cinder] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:21.875086 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cfe756-6096-429f-b8af-c4f80db91eee service cinder] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:21.875370 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cfe756-6096-429f-b8af-c4f80db91eee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:21.885591 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cfe756-6096-429f-b8af-c4f80db91eee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:21.887068 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cfe756-6096-429f-b8af-c4f80db91eee service cinder] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:21.897338 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1426/2811] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:52:21 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:52:21.961401 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4c58e4-970d-47b3-81cf-c1b63936ca90 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:21.966825 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480092f1-c2a2-4ca3-9097-1063c9be49b5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6330294b1c8e4dcd85aad189f007b0aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:21.967485 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480092f1-c2a2-4ca3-9097-1063c9be49b5 service placement] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:21.967765 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480092f1-c2a2-4ca3-9097-1063c9be49b5 service placement] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:21.968040 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480092f1-c2a2-4ca3-9097-1063c9be49b5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:21.970678 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-480092f1-c2a2-4ca3-9097-1063c9be49b5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:52:21.980385 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:52:21.982945 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480092f1-c2a2-4ca3-9097-1063c9be49b5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:21.984693 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480092f1-c2a2-4ca3-9097-1063c9be49b5 service placement] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:21.994588 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1386/2812] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:52:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:52:23.080208 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cfe756-6096-429f-b8af-c4f80db91eee service cinder] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:23.145401 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cfe756-6096-429f-b8af-c4f80db91eee service cinder] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:23.150147 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d82bb7-971a-40e1-968b-c157ec10cfa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:23.151413 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d82bb7-971a-40e1-968b-c157ec10cfa8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:23.151890 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d82bb7-971a-40e1-968b-c157ec10cfa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:23.153024 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d82bb7-971a-40e1-968b-c157ec10cfa8 admin admin] PATH_INFO: `/v3/users/412a237e9c4b4c15bfb688a07a57d114` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:23.153993 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d82bb7-971a-40e1-968b-c157ec10cfa8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=412a237e9c4b4c15bfb688a07a57d114)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:23.156028 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d82bb7-971a-40e1-968b-c157ec10cfa8 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:23.173464 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:52:23.327995 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-23d82bb7-971a-40e1-968b-c157ec10cfa8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '412a237e9c4b4c15bfb688a07a57d114'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:23.328692 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-23d82bb7-971a-40e1-968b-c157ec10cfa8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '412a237e9c4b4c15bfb688a07a57d114'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:23.329319 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-23d82bb7-971a-40e1-968b-c157ec10cfa8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '412a237e9c4b4c15bfb688a07a57d114'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:23.336109 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-23d82bb7-971a-40e1-968b-c157ec10cfa8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '412a237e9c4b4c15bfb688a07a57d114'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:23.348643 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-23d82bb7-971a-40e1-968b-c157ec10cfa8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '412a237e9c4b4c15bfb688a07a57d114'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:23.428684 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1427/2813] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:52:23 2022] DELETE /identity/v3/users/412a237e9c4b4c15bfb688a07a57d114 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:52:23.433432 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c467d4-2698-425b-9f96-a162c4ea7de7 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:23.433968 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c467d4-2698-425b-9f96-a162c4ea7de7 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:23.433968 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c467d4-2698-425b-9f96-a162c4ea7de7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:23.456545 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-80c467d4-2698-425b-9f96-a162c4ea7de7 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:52:23.456545 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-80c467d4-2698-425b-9f96-a162c4ea7de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:52:23.512387 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1387/2814] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:52:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:52:23.522656 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d82bb7-971a-40e1-968b-c157ec10cfa8 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:23.524057 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-23d82bb7-971a-40e1-968b-c157ec10cfa8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:52:23.590567 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d82bb7-971a-40e1-968b-c157ec10cfa8 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:23.594966 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159dd59d-11d0-4c68-b430-9fdd7eec4abe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:23.595740 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159dd59d-11d0-4c68-b430-9fdd7eec4abe service neutron] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:23.596023 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159dd59d-11d0-4c68-b430-9fdd7eec4abe service neutron] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:23.596303 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159dd59d-11d0-4c68-b430-9fdd7eec4abe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:23.606611 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159dd59d-11d0-4c68-b430-9fdd7eec4abe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:23.608120 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159dd59d-11d0-4c68-b430-9fdd7eec4abe service neutron] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:23.618111 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1428/2815] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:52:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:52:24.058684 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c467d4-2698-425b-9f96-a162c4ea7de7 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:24.060441 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-80c467d4-2698-425b-9f96-a162c4ea7de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:52:24.073767 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c467d4-2698-425b-9f96-a162c4ea7de7 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:24.078677 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc91cf2-41d5-474f-853b-d2fc41e4fa39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:24.079921 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc91cf2-41d5-474f-853b-d2fc41e4fa39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:24.087036 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc91cf2-41d5-474f-853b-d2fc41e4fa39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:24.087036 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc91cf2-41d5-474f-853b-d2fc41e4fa39 admin admin] PATH_INFO: `/v3/projects/c25d90665fb548d79ea3902ec50dc236` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:24.087036 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc91cf2-41d5-474f-853b-d2fc41e4fa39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c25d90665fb548d79ea3902ec50dc236)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:24.087036 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc91cf2-41d5-474f-853b-d2fc41e4fa39 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:24.208546 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-dbc91cf2-41d5-474f-853b-d2fc41e4fa39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c25d90665fb548d79ea3902ec50dc236', 'request_id': 'req-dbc91cf2-41d5-474f-853b-d2fc41e4fa39'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:24.308696 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-dbc91cf2-41d5-474f-853b-d2fc41e4fa39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c25d90665fb548d79ea3902ec50dc236', 'request_id': 'req-dbc91cf2-41d5-474f-853b-d2fc41e4fa39'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:24.315995 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-dbc91cf2-41d5-474f-853b-d2fc41e4fa39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c25d90665fb548d79ea3902ec50dc236', 'request_id': 'req-dbc91cf2-41d5-474f-853b-d2fc41e4fa39'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:24.328425 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-dbc91cf2-41d5-474f-853b-d2fc41e4fa39 admin admin] The token cache is being invalidate because project c25d90665fb548d79ea3902ec50dc236 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:52:24.328723 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-dbc91cf2-41d5-474f-853b-d2fc41e4fa39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:24.329545 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-dbc91cf2-41d5-474f-853b-d2fc41e4fa39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:24.331132 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1388/2816] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:52:24 2022] DELETE /identity/v3/projects/c25d90665fb548d79ea3902ec50dc236 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:52:24.335254 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159dd59d-11d0-4c68-b430-9fdd7eec4abe service neutron] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:24.345252 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159dd59d-11d0-4c68-b430-9fdd7eec4abe service neutron] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:24.405405 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1557311-a8cc-4ee9-8175-42b5fc81fb05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:24.406816 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1557311-a8cc-4ee9-8175-42b5fc81fb05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:24.407079 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1557311-a8cc-4ee9-8175-42b5fc81fb05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:24.407354 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1557311-a8cc-4ee9-8175-42b5fc81fb05 admin admin] PATH_INFO: `/v3/users/178b954bafb54fbf95dd39e738cb3b79` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:24.408653 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1557311-a8cc-4ee9-8175-42b5fc81fb05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=178b954bafb54fbf95dd39e738cb3b79)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:24.410626 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1557311-a8cc-4ee9-8175-42b5fc81fb05 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:24.598949 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:52:24.607577 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-f1557311-a8cc-4ee9-8175-42b5fc81fb05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '178b954bafb54fbf95dd39e738cb3b79'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:24.608216 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-f1557311-a8cc-4ee9-8175-42b5fc81fb05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '178b954bafb54fbf95dd39e738cb3b79'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:24.608894 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-f1557311-a8cc-4ee9-8175-42b5fc81fb05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '178b954bafb54fbf95dd39e738cb3b79'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:24.616806 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-f1557311-a8cc-4ee9-8175-42b5fc81fb05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '178b954bafb54fbf95dd39e738cb3b79'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:24.637093 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-f1557311-a8cc-4ee9-8175-42b5fc81fb05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '178b954bafb54fbf95dd39e738cb3b79'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:24.721015 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1429/2817] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:52:24 2022] DELETE /identity/v3/users/178b954bafb54fbf95dd39e738cb3b79 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:52:25.167225 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc91cf2-41d5-474f-853b-d2fc41e4fa39 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:25.168794 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-dbc91cf2-41d5-474f-853b-d2fc41e4fa39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:52:25.182519 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc91cf2-41d5-474f-853b-d2fc41e4fa39 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:25.248529 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0ec8dd-e846-4083-9ea0-67d52124f482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:25.250163 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0ec8dd-e846-4083-9ea0-67d52124f482 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:25.250736 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0ec8dd-e846-4083-9ea0-67d52124f482 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:25.250951 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0ec8dd-e846-4083-9ea0-67d52124f482 admin admin] PATH_INFO: `/v3/projects/b6084bad35434820ab0ecefe99b95e4e` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:25.252205 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0ec8dd-e846-4083-9ea0-67d52124f482 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6084bad35434820ab0ecefe99b95e4e)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:25.254364 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0ec8dd-e846-4083-9ea0-67d52124f482 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:25.282423 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:52:25.377149 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-1d0ec8dd-e846-4083-9ea0-67d52124f482 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6084bad35434820ab0ecefe99b95e4e', 'request_id': 'req-1d0ec8dd-e846-4083-9ea0-67d52124f482'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:25.455236 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-1d0ec8dd-e846-4083-9ea0-67d52124f482 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6084bad35434820ab0ecefe99b95e4e', 'request_id': 'req-1d0ec8dd-e846-4083-9ea0-67d52124f482'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:25.461400 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-1d0ec8dd-e846-4083-9ea0-67d52124f482 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6084bad35434820ab0ecefe99b95e4e', 'request_id': 'req-1d0ec8dd-e846-4083-9ea0-67d52124f482'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:25.473892 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-1d0ec8dd-e846-4083-9ea0-67d52124f482 admin admin] The token cache is being invalidate because project b6084bad35434820ab0ecefe99b95e4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:52:25.474190 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-1d0ec8dd-e846-4083-9ea0-67d52124f482 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:25.474875 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-1d0ec8dd-e846-4083-9ea0-67d52124f482 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:25.476491 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1389/2818] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:52:25 2022] DELETE /identity/v3/projects/b6084bad35434820ab0ecefe99b95e4e => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:52:25.481467 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc156dd9-d2b8-4ba6-94cc-dcdc7619afbb None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:25.481983 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc156dd9-d2b8-4ba6-94cc-dcdc7619afbb None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:25.482080 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc156dd9-d2b8-4ba6-94cc-dcdc7619afbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:25.501835 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-cc156dd9-d2b8-4ba6-94cc-dcdc7619afbb None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:52:25.503074 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-cc156dd9-d2b8-4ba6-94cc-dcdc7619afbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:52:25.567103 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1430/2819] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:52:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:52:25.576057 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca2686d-cdc2-438d-b1c3-d0f157520a3c None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:25.576354 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca2686d-cdc2-438d-b1c3-d0f157520a3c None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:25.576664 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca2686d-cdc2-438d-b1c3-d0f157520a3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:25.595769 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-2ca2686d-cdc2-438d-b1c3-d0f157520a3c None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:52:25.597034 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-2ca2686d-cdc2-438d-b1c3-d0f157520a3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:52:25.604951 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1390/2820] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:52:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:52:25.609804 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc156dd9-d2b8-4ba6-94cc-dcdc7619afbb None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:25.619799 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc156dd9-d2b8-4ba6-94cc-dcdc7619afbb None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:25.624707 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c40d6ec-2bea-48e6-b9b4-bde760d2e53c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:25.625885 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c40d6ec-2bea-48e6-b9b4-bde760d2e53c admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:25.626167 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c40d6ec-2bea-48e6-b9b4-bde760d2e53c admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:25.626439 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c40d6ec-2bea-48e6-b9b4-bde760d2e53c admin admin] PATH_INFO: `/v3/domains` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:25.627176 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c40d6ec-2bea-48e6-b9b4-bde760d2e53c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:52:25.627480 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c40d6ec-2bea-48e6-b9b4-bde760d2e53c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:25.629115 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c40d6ec-2bea-48e6-b9b4-bde760d2e53c admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:25.641143 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1431/2821] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jun 7 14:52:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:52:25.645629 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca2686d-cdc2-438d-b1c3-d0f157520a3c None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:25.656015 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca2686d-cdc2-438d-b1c3-d0f157520a3c None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:25.660780 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d15c3c-ae80-4d82-b3a8-784ff15673b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:25.661451 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d15c3c-ae80-4d82-b3a8-784ff15673b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:25.661738 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d15c3c-ae80-4d82-b3a8-784ff15673b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:25.662018 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d15c3c-ae80-4d82-b3a8-784ff15673b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:25.662720 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d15c3c-ae80-4d82-b3a8-784ff15673b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:25.664635 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d15c3c-ae80-4d82-b3a8-784ff15673b2 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:25.807049 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1391/2822] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:52:25 2022] POST /identity/v3/projects => generated 359 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:52:25.811294 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c40d6ec-2bea-48e6-b9b4-bde760d2e53c admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:25.821994 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c40d6ec-2bea-48e6-b9b4-bde760d2e53c admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:25.879415 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e411b3-4bd2-48ee-ad2c-59b1b937ebf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:25.880095 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e411b3-4bd2-48ee-ad2c-59b1b937ebf6 admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:25.880396 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e411b3-4bd2-48ee-ad2c-59b1b937ebf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:25.880690 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e411b3-4bd2-48ee-ad2c-59b1b937ebf6 admin admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:25.881397 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e411b3-4bd2-48ee-ad2c-59b1b937ebf6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:25.883345 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e411b3-4bd2-48ee-ad2c-59b1b937ebf6 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:25.975657 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1432/2823] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:52:25 2022] POST /identity/v3/users => generated 280 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:52:25.979941 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d15c3c-ae80-4d82-b3a8-784ff15673b2 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:25.992341 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d15c3c-ae80-4d82-b3a8-784ff15673b2 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:25.997259 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29214a59-7ffb-46f1-9cc7-672765146b0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:25.997952 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29214a59-7ffb-46f1-9cc7-672765146b0e admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:25.998263 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29214a59-7ffb-46f1-9cc7-672765146b0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:25.998536 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29214a59-7ffb-46f1-9cc7-672765146b0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:25.999272 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29214a59-7ffb-46f1-9cc7-672765146b0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:26.001441 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29214a59-7ffb-46f1-9cc7-672765146b0e admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:26.016435 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1392/2824] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:52:25 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:52:26.021743 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e411b3-4bd2-48ee-ad2c-59b1b937ebf6 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:26.033534 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e411b3-4bd2-48ee-ad2c-59b1b937ebf6 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:26.038282 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb984eef-dab9-43a4-b702-40d86f8f7d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:26.038951 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb984eef-dab9-43a4-b702-40d86f8f7d41 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:26.039233 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb984eef-dab9-43a4-b702-40d86f8f7d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:26.039505 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb984eef-dab9-43a4-b702-40d86f8f7d41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:26.040231 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb984eef-dab9-43a4-b702-40d86f8f7d41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:26.041870 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb984eef-dab9-43a4-b702-40d86f8f7d41 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:26.052261 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1433/2825] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:52:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:52:26.056716 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29214a59-7ffb-46f1-9cc7-672765146b0e admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:26.070055 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29214a59-7ffb-46f1-9cc7-672765146b0e admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:26.075435 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386338ac-13cc-44b6-9994-2d2094651e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:26.076355 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386338ac-13cc-44b6-9994-2d2094651e0e admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:26.076675 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386338ac-13cc-44b6-9994-2d2094651e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:26.076996 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386338ac-13cc-44b6-9994-2d2094651e0e admin admin] PATH_INFO: `/v3/projects/fe274141239047acad428a62a968787c/users/5f6e3b5e2bc749838e5151b744d46ed8/roles/3771341b3be641139e752aab690397a4` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:26.093212 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386338ac-13cc-44b6-9994-2d2094651e0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe274141239047acad428a62a968787c, user_id=5f6e3b5e2bc749838e5151b744d46ed8, role_id=3771341b3be641139e752aab690397a4)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:26.095898 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386338ac-13cc-44b6-9994-2d2094651e0e admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:26.176618 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1393/2826] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:52:26 2022] PUT /identity/v3/projects/fe274141239047acad428a62a968787c/users/5f6e3b5e2bc749838e5151b744d46ed8/roles/3771341b3be641139e752aab690397a4 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:52:26.186842 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f9b1ab-2534-4468-a327-0b29dc40fa80 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:26.187640 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f9b1ab-2534-4468-a327-0b29dc40fa80 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:26.187925 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f9b1ab-2534-4468-a327-0b29dc40fa80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:26.219374 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-d3f9b1ab-2534-4468-a327-0b29dc40fa80 None None] MFA Rules not processed for user `5f6e3b5e2bc749838e5151b744d46ed8`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:52:26.223882 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-d3f9b1ab-2534-4468-a327-0b29dc40fa80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:52:26.236942 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:52:26.297192 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1434/2827] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:52:26 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:52:26.337462 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386338ac-13cc-44b6-9994-2d2094651e0e admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:26.339723 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-386338ac-13cc-44b6-9994-2d2094651e0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:52:26.406412 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386338ac-13cc-44b6-9994-2d2094651e0e admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:26.410682 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b87fd9-a3c7-48e3-a7bc-3427b40b3d68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d81f597cb44c56a966537089bcce4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:26.411337 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b87fd9-a3c7-48e3-a7bc-3427b40b3d68 service cinder] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:26.411792 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b87fd9-a3c7-48e3-a7bc-3427b40b3d68 service cinder] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:26.412079 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b87fd9-a3c7-48e3-a7bc-3427b40b3d68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:26.424147 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b87fd9-a3c7-48e3-a7bc-3427b40b3d68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:26.425616 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b87fd9-a3c7-48e3-a7bc-3427b40b3d68 service cinder] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:26.435350 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1394/2828] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:52:26 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:52:26.452742 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f9b1ab-2534-4468-a327-0b29dc40fa80 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:26.558552 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f9b1ab-2534-4468-a327-0b29dc40fa80 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:26.564046 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86050d31-db28-4af8-b083-0e4150c10467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:26.564973 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86050d31-db28-4af8-b083-0e4150c10467 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:26.565262 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86050d31-db28-4af8-b083-0e4150c10467 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:26.565544 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86050d31-db28-4af8-b083-0e4150c10467 admin admin] PATH_INFO: `/v3/users/5f6e3b5e2bc749838e5151b744d46ed8` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:26.566773 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86050d31-db28-4af8-b083-0e4150c10467 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f6e3b5e2bc749838e5151b744d46ed8)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:26.568993 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86050d31-db28-4af8-b083-0e4150c10467 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:26.836664 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-86050d31-db28-4af8-b083-0e4150c10467 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f6e3b5e2bc749838e5151b744d46ed8'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:26.837303 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-86050d31-db28-4af8-b083-0e4150c10467 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f6e3b5e2bc749838e5151b744d46ed8'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:26.838083 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-86050d31-db28-4af8-b083-0e4150c10467 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f6e3b5e2bc749838e5151b744d46ed8'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:26.846420 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-86050d31-db28-4af8-b083-0e4150c10467 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f6e3b5e2bc749838e5151b744d46ed8'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:26.858467 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-86050d31-db28-4af8-b083-0e4150c10467 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f6e3b5e2bc749838e5151b744d46ed8'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:26.937084 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1435/2829] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:52:26 2022] DELETE /identity/v3/users/5f6e3b5e2bc749838e5151b744d46ed8 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:52:26.943005 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40743265-8a60-4285-9978-c4d7a2fac734 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:26.943328 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40743265-8a60-4285-9978-c4d7a2fac734 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:26.943615 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40743265-8a60-4285-9978-c4d7a2fac734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:26.962502 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-40743265-8a60-4285-9978-c4d7a2fac734 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:52:26.963722 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-40743265-8a60-4285-9978-c4d7a2fac734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:52:27.011400 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:52:27.027933 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1395/2830] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:52:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:52:27.038953 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86050d31-db28-4af8-b083-0e4150c10467 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:27.040763 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-86050d31-db28-4af8-b083-0e4150c10467 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:52:27.077020 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:52:27.100766 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86050d31-db28-4af8-b083-0e4150c10467 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:27.104280 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64352779-e648-48c6-98c7-4f4f473b944e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:27.104969 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64352779-e648-48c6-98c7-4f4f473b944e service neutron] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:27.105253 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64352779-e648-48c6-98c7-4f4f473b944e service neutron] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:27.105533 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64352779-e648-48c6-98c7-4f4f473b944e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:27.114446 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64352779-e648-48c6-98c7-4f4f473b944e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:27.116903 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64352779-e648-48c6-98c7-4f4f473b944e service neutron] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:27.125237 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1436/2831] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:52:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:52:27.577532 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40743265-8a60-4285-9978-c4d7a2fac734 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:27.580717 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-40743265-8a60-4285-9978-c4d7a2fac734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:52:27.596767 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40743265-8a60-4285-9978-c4d7a2fac734 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:27.602125 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a860ae-a252-4cd4-b723-5d534dbd3cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:27.602938 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a860ae-a252-4cd4-b723-5d534dbd3cea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:27.603231 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a860ae-a252-4cd4-b723-5d534dbd3cea admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:27.603517 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a860ae-a252-4cd4-b723-5d534dbd3cea admin admin] PATH_INFO: `/v3/projects/fe274141239047acad428a62a968787c` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:27.605011 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a860ae-a252-4cd4-b723-5d534dbd3cea admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe274141239047acad428a62a968787c)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:27.606908 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a860ae-a252-4cd4-b723-5d534dbd3cea admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:27.769925 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-e6a860ae-a252-4cd4-b723-5d534dbd3cea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe274141239047acad428a62a968787c', 'request_id': 'req-e6a860ae-a252-4cd4-b723-5d534dbd3cea'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:27.849215 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-e6a860ae-a252-4cd4-b723-5d534dbd3cea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe274141239047acad428a62a968787c', 'request_id': 'req-e6a860ae-a252-4cd4-b723-5d534dbd3cea'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:27.856249 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-e6a860ae-a252-4cd4-b723-5d534dbd3cea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe274141239047acad428a62a968787c', 'request_id': 'req-e6a860ae-a252-4cd4-b723-5d534dbd3cea'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:27.868937 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-e6a860ae-a252-4cd4-b723-5d534dbd3cea admin admin] The token cache is being invalidate because project fe274141239047acad428a62a968787c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:52:27.869193 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-e6a860ae-a252-4cd4-b723-5d534dbd3cea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:27.869815 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-e6a860ae-a252-4cd4-b723-5d534dbd3cea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:27.871457 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1396/2832] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:52:27 2022] DELETE /identity/v3/projects/fe274141239047acad428a62a968787c => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:52:27.877028 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6898830-0c11-4f8b-bcfd-a1be3a6857ae None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:27.877541 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6898830-0c11-4f8b-bcfd-a1be3a6857ae None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:27.877711 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6898830-0c11-4f8b-bcfd-a1be3a6857ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:27.897697 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-e6898830-0c11-4f8b-bcfd-a1be3a6857ae None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:52:27.898964 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-e6898830-0c11-4f8b-bcfd-a1be3a6857ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:52:27.974354 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1437/2833] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:52:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:52:27.983768 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8399b4fc-f457-4ace-821f-1248bfd8a61f None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:27.984710 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8399b4fc-f457-4ace-821f-1248bfd8a61f None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:27.985007 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8399b4fc-f457-4ace-821f-1248bfd8a61f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:28.007784 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-8399b4fc-f457-4ace-821f-1248bfd8a61f None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:52:28.009027 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-8399b4fc-f457-4ace-821f-1248bfd8a61f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:52:28.020181 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1397/2834] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:52:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:52:28.024921 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6898830-0c11-4f8b-bcfd-a1be3a6857ae None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:28.035934 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6898830-0c11-4f8b-bcfd-a1be3a6857ae None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:28.041054 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaf8b09-dfb4-48c8-935c-9bf870bb6b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:28.041806 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaf8b09-dfb4-48c8-935c-9bf870bb6b1f admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:28.042036 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaf8b09-dfb4-48c8-935c-9bf870bb6b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:28.042480 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaf8b09-dfb4-48c8-935c-9bf870bb6b1f admin admin] PATH_INFO: `/v3/domains` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:28.043221 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaf8b09-dfb4-48c8-935c-9bf870bb6b1f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80387) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:52:28.043525 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaf8b09-dfb4-48c8-935c-9bf870bb6b1f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:28.045494 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaf8b09-dfb4-48c8-935c-9bf870bb6b1f admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:28.062917 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1438/2835] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jun 7 14:52:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 14:52:28.067465 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8399b4fc-f457-4ace-821f-1248bfd8a61f None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:28.076815 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8399b4fc-f457-4ace-821f-1248bfd8a61f None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:28.081652 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35df1eab-5287-4c18-a222-7b0e51af318c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:28.082317 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35df1eab-5287-4c18-a222-7b0e51af318c admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:28.082648 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35df1eab-5287-4c18-a222-7b0e51af318c admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:28.082932 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35df1eab-5287-4c18-a222-7b0e51af318c admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:28.083630 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35df1eab-5287-4c18-a222-7b0e51af318c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:28.085567 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35df1eab-5287-4c18-a222-7b0e51af318c admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:28.185378 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1398/2836] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:52:28 2022] POST /identity/v3/projects => generated 373 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:52:28.189939 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaf8b09-dfb4-48c8-935c-9bf870bb6b1f admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:28.201722 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaf8b09-dfb4-48c8-935c-9bf870bb6b1f admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:28.264541 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d796cf7-a5fa-4d50-8fed-3a475df1ba2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:28.265228 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d796cf7-a5fa-4d50-8fed-3a475df1ba2f admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:28.265519 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d796cf7-a5fa-4d50-8fed-3a475df1ba2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:28.265801 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d796cf7-a5fa-4d50-8fed-3a475df1ba2f admin admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:28.266502 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d796cf7-a5fa-4d50-8fed-3a475df1ba2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:28.268464 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d796cf7-a5fa-4d50-8fed-3a475df1ba2f admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:28.395532 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1439/2837] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:52:28 2022] POST /identity/v3/users => generated 287 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:52:28.399151 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35df1eab-5287-4c18-a222-7b0e51af318c admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:28.410499 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35df1eab-5287-4c18-a222-7b0e51af318c admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:28.415159 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee81220-e3e1-459d-a9c5-283540e7fe5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:28.415829 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee81220-e3e1-459d-a9c5-283540e7fe5a admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:28.416119 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee81220-e3e1-459d-a9c5-283540e7fe5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:28.416432 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee81220-e3e1-459d-a9c5-283540e7fe5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:28.417179 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee81220-e3e1-459d-a9c5-283540e7fe5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:28.418778 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee81220-e3e1-459d-a9c5-283540e7fe5a admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:28.430412 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1399/2838] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:52:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:52:28.441667 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d796cf7-a5fa-4d50-8fed-3a475df1ba2f admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:28.451877 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d796cf7-a5fa-4d50-8fed-3a475df1ba2f admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:28.456631 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cb96f5-a7c4-4f38-9a83-0de3f9feefc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:28.457313 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cb96f5-a7c4-4f38-9a83-0de3f9feefc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:28.457568 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cb96f5-a7c4-4f38-9a83-0de3f9feefc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:28.457841 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cb96f5-a7c4-4f38-9a83-0de3f9feefc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:28.458566 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cb96f5-a7c4-4f38-9a83-0de3f9feefc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:28.460287 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cb96f5-a7c4-4f38-9a83-0de3f9feefc3 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:28.471925 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1440/2839] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:52:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:52:28.479352 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee81220-e3e1-459d-a9c5-283540e7fe5a admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:28.490774 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee81220-e3e1-459d-a9c5-283540e7fe5a admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:28.495822 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7991566f-58a3-471d-8381-1dd889a32162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:28.499300 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7991566f-58a3-471d-8381-1dd889a32162 admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:28.499300 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7991566f-58a3-471d-8381-1dd889a32162 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:28.499300 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7991566f-58a3-471d-8381-1dd889a32162 admin admin] PATH_INFO: `/v3/projects/005732c774f145ffadd2a7bcb3049f26/users/d2228e3ceb4c4f94bf4207ee9a8aac98/roles/3771341b3be641139e752aab690397a4` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:28.513705 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7991566f-58a3-471d-8381-1dd889a32162 admin admin] RBAC: Authorizing `identity:create_grant(project_id=005732c774f145ffadd2a7bcb3049f26, user_id=d2228e3ceb4c4f94bf4207ee9a8aac98, role_id=3771341b3be641139e752aab690397a4)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:28.517201 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7991566f-58a3-471d-8381-1dd889a32162 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:28.603776 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1400/2840] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:52:28 2022] PUT /identity/v3/projects/005732c774f145ffadd2a7bcb3049f26/users/d2228e3ceb4c4f94bf4207ee9a8aac98/roles/3771341b3be641139e752aab690397a4 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:52:28.609015 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1093ebe4-99f3-42b6-841f-2351193d01b1 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:28.609430 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1093ebe4-99f3-42b6-841f-2351193d01b1 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:28.609842 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1093ebe4-99f3-42b6-841f-2351193d01b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:28.628962 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-1093ebe4-99f3-42b6-841f-2351193d01b1 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:52:28.630183 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-1093ebe4-99f3-42b6-841f-2351193d01b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:52:28.665458 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:52:28.689893 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1441/2841] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:52:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:52:28.699391 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7991566f-58a3-471d-8381-1dd889a32162 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:28.701202 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-7991566f-58a3-471d-8381-1dd889a32162 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:52:28.759611 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7991566f-58a3-471d-8381-1dd889a32162 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:28.763310 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb247bd-2dc2-4bf8-b36f-4f9243989322 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:28.763961 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb247bd-2dc2-4bf8-b36f-4f9243989322 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:28.764248 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb247bd-2dc2-4bf8-b36f-4f9243989322 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:28.764560 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb247bd-2dc2-4bf8-b36f-4f9243989322 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:28.774295 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb247bd-2dc2-4bf8-b36f-4f9243989322 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:28.775746 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb247bd-2dc2-4bf8-b36f-4f9243989322 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:28.786629 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1401/2842] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:52:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:52:30.187171 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d0e66c-df3f-4148-869c-40fc56e65ffc None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:30.187171 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d0e66c-df3f-4148-869c-40fc56e65ffc None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:30.187171 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d0e66c-df3f-4148-869c-40fc56e65ffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:30.215831 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-63d0e66c-df3f-4148-869c-40fc56e65ffc None None] MFA Rules not processed for user `d2228e3ceb4c4f94bf4207ee9a8aac98`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:52:30.219343 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-63d0e66c-df3f-4148-869c-40fc56e65ffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:52:30.296959 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1442/2843] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:52:30 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:52:30.307622 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb247bd-2dc2-4bf8-b36f-4f9243989322 service neutron] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:30.309126 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-9fb247bd-2dc2-4bf8-b36f-4f9243989322 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:52:30.322946 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:52:30.368230 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb247bd-2dc2-4bf8-b36f-4f9243989322 service neutron] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:30.371985 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0d345b-d39d-4cb1-b927-4ce8200bf795 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89d81f597cb44c56a966537089bcce4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:30.372700 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0d345b-d39d-4cb1-b927-4ce8200bf795 service cinder] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:30.372990 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0d345b-d39d-4cb1-b927-4ce8200bf795 service cinder] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:30.373386 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0d345b-d39d-4cb1-b927-4ce8200bf795 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:30.386803 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0d345b-d39d-4cb1-b927-4ce8200bf795 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:30.388261 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0d345b-d39d-4cb1-b927-4ce8200bf795 service cinder] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:30.398171 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1402/2844] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:52:30 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:52:33.350730 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41f73aa-b46b-46e8-a450-0dfb0c65eeb0 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:33.351314 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41f73aa-b46b-46e8-a450-0dfb0c65eeb0 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:33.351314 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41f73aa-b46b-46e8-a450-0dfb0c65eeb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:33.369686 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-d41f73aa-b46b-46e8-a450-0dfb0c65eeb0 None None] MFA Rules not processed for user `d2228e3ceb4c4f94bf4207ee9a8aac98`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:52:33.370892 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-d41f73aa-b46b-46e8-a450-0dfb0c65eeb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:52:33.378231 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1443/2845] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:52:33 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:52:40.465970 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=21 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:52:43.364292 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=44 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:52:52.811708 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0d345b-d39d-4cb1-b927-4ce8200bf795 service cinder] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:52.813362 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-9d0d345b-d39d-4cb1-b927-4ce8200bf795 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:52:52.827230 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0d345b-d39d-4cb1-b927-4ce8200bf795 service cinder] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:52.831762 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a917e63a-157a-4db7-90d1-f8443f37d219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:52.832803 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a917e63a-157a-4db7-90d1-f8443f37d219 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:52.833093 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a917e63a-157a-4db7-90d1-f8443f37d219 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:52.833371 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a917e63a-157a-4db7-90d1-f8443f37d219 admin admin] PATH_INFO: `/v3/users/f46eae035e894a09a6eaced29cc5c43a` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:52.834547 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a917e63a-157a-4db7-90d1-f8443f37d219 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f46eae035e894a09a6eaced29cc5c43a)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:52.836483 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a917e63a-157a-4db7-90d1-f8443f37d219 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:53.088212 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a917e63a-157a-4db7-90d1-f8443f37d219 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f46eae035e894a09a6eaced29cc5c43a'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:53.088912 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a917e63a-157a-4db7-90d1-f8443f37d219 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f46eae035e894a09a6eaced29cc5c43a'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:53.089599 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a917e63a-157a-4db7-90d1-f8443f37d219 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f46eae035e894a09a6eaced29cc5c43a'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:53.096581 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a917e63a-157a-4db7-90d1-f8443f37d219 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f46eae035e894a09a6eaced29cc5c43a'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:53.117388 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a917e63a-157a-4db7-90d1-f8443f37d219 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f46eae035e894a09a6eaced29cc5c43a'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:53.205588 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1403/2846] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:52:52 2022] DELETE /identity/v3/users/f46eae035e894a09a6eaced29cc5c43a => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:52:53.210334 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84282ee-77b5-473f-b6f0-0afd00360414 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:53.210845 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84282ee-77b5-473f-b6f0-0afd00360414 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:53.211002 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84282ee-77b5-473f-b6f0-0afd00360414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:53.228558 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-f84282ee-77b5-473f-b6f0-0afd00360414 None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:52:53.232640 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-f84282ee-77b5-473f-b6f0-0afd00360414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:52:53.290135 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1444/2847] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:52:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:52:53.299739 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a917e63a-157a-4db7-90d1-f8443f37d219 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:53.301177 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-a917e63a-157a-4db7-90d1-f8443f37d219 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:52:53.359331 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a917e63a-157a-4db7-90d1-f8443f37d219 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:53.362992 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df85267-d844-4112-a351-c22495f16c0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:53.364245 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df85267-d844-4112-a351-c22495f16c0e service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:53.364573 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df85267-d844-4112-a351-c22495f16c0e service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:53.364854 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df85267-d844-4112-a351-c22495f16c0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:53.373811 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df85267-d844-4112-a351-c22495f16c0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:53.375246 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df85267-d844-4112-a351-c22495f16c0e service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:53.385559 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1404/2848] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:52:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:52:53.839226 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84282ee-77b5-473f-b6f0-0afd00360414 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:53.841631 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-f84282ee-77b5-473f-b6f0-0afd00360414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:52:53.855109 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84282ee-77b5-473f-b6f0-0afd00360414 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:53.861742 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c8c9cf-4222-40c0-bd47-8511fb7a6bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:53.862629 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c8c9cf-4222-40c0-bd47-8511fb7a6bd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:53.862917 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c8c9cf-4222-40c0-bd47-8511fb7a6bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:53.863201 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c8c9cf-4222-40c0-bd47-8511fb7a6bd7 admin admin] PATH_INFO: `/v3/projects/2581b55726844ce1b36f318ca0e8259d` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:53.865186 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c8c9cf-4222-40c0-bd47-8511fb7a6bd7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2581b55726844ce1b36f318ca0e8259d)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:53.866956 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c8c9cf-4222-40c0-bd47-8511fb7a6bd7 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:54.005510 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-03c8c9cf-4222-40c0-bd47-8511fb7a6bd7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2581b55726844ce1b36f318ca0e8259d', 'request_id': 'req-03c8c9cf-4222-40c0-bd47-8511fb7a6bd7'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:54.079581 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-03c8c9cf-4222-40c0-bd47-8511fb7a6bd7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2581b55726844ce1b36f318ca0e8259d', 'request_id': 'req-03c8c9cf-4222-40c0-bd47-8511fb7a6bd7'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:54.085527 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-03c8c9cf-4222-40c0-bd47-8511fb7a6bd7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2581b55726844ce1b36f318ca0e8259d', 'request_id': 'req-03c8c9cf-4222-40c0-bd47-8511fb7a6bd7'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:54.096860 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-03c8c9cf-4222-40c0-bd47-8511fb7a6bd7 admin admin] The token cache is being invalidate because project 2581b55726844ce1b36f318ca0e8259d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:52:54.097169 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-03c8c9cf-4222-40c0-bd47-8511fb7a6bd7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:54.097805 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-03c8c9cf-4222-40c0-bd47-8511fb7a6bd7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:52:54.099376 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1445/2849] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:52:53 2022] DELETE /identity/v3/projects/2581b55726844ce1b36f318ca0e8259d => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:52:54.104300 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae7a88d-4916-46af-b4ec-b1e3f9eca9ef None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:54.104849 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae7a88d-4916-46af-b4ec-b1e3f9eca9ef None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:54.105101 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae7a88d-4916-46af-b4ec-b1e3f9eca9ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:54.123283 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-2ae7a88d-4916-46af-b4ec-b1e3f9eca9ef None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:52:54.124556 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-2ae7a88d-4916-46af-b4ec-b1e3f9eca9ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:52:54.185809 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1405/2850] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:52:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:52:54.194287 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131fcf7f-ff13-4d5d-acb5-79c05ac7559b None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:54.194596 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131fcf7f-ff13-4d5d-acb5-79c05ac7559b None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:54.194859 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131fcf7f-ff13-4d5d-acb5-79c05ac7559b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:54.212767 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-131fcf7f-ff13-4d5d-acb5-79c05ac7559b None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:52:54.215279 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-131fcf7f-ff13-4d5d-acb5-79c05ac7559b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:52:54.221659 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1446/2851] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:52:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:52:54.226275 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae7a88d-4916-46af-b4ec-b1e3f9eca9ef None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:54.236127 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae7a88d-4916-46af-b4ec-b1e3f9eca9ef None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:54.241422 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ef6530-acd1-494d-a19a-a900397de1a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:54.244102 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ef6530-acd1-494d-a19a-a900397de1a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:54.244102 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ef6530-acd1-494d-a19a-a900397de1a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:54.244102 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ef6530-acd1-494d-a19a-a900397de1a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:54.244102 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ef6530-acd1-494d-a19a-a900397de1a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 14:52:54.244102 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ef6530-acd1-494d-a19a-a900397de1a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:54.245247 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ef6530-acd1-494d-a19a-a900397de1a4 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:54.256940 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1406/2852] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jun 7 14:52:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 14:52:54.261173 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131fcf7f-ff13-4d5d-acb5-79c05ac7559b None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:54.270685 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131fcf7f-ff13-4d5d-acb5-79c05ac7559b None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:54.275519 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff64c5da-e41f-4485-b668-c8db12fba3b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:54.276171 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff64c5da-e41f-4485-b668-c8db12fba3b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:54.276565 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff64c5da-e41f-4485-b668-c8db12fba3b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:54.276850 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff64c5da-e41f-4485-b668-c8db12fba3b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:54.277541 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff64c5da-e41f-4485-b668-c8db12fba3b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:54.279758 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff64c5da-e41f-4485-b668-c8db12fba3b9 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:54.389674 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1447/2853] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:52:54 2022] POST /identity/v3/projects => generated 373 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:52:54.394263 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ef6530-acd1-494d-a19a-a900397de1a4 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:54.400055 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:52:54.405270 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ef6530-acd1-494d-a19a-a900397de1a4 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:54.460497 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3c3c67-6d38-4503-bf03-966a36c2cc51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:54.461279 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3c3c67-6d38-4503-bf03-966a36c2cc51 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:54.461549 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3c3c67-6d38-4503-bf03-966a36c2cc51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:54.461829 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3c3c67-6d38-4503-bf03-966a36c2cc51 admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:54.462520 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3c3c67-6d38-4503-bf03-966a36c2cc51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:54.464471 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3c3c67-6d38-4503-bf03-966a36c2cc51 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:54.582343 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1407/2854] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:52:54 2022] POST /identity/v3/users => generated 287 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:52:54.589853 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff64c5da-e41f-4485-b668-c8db12fba3b9 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:54.599980 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff64c5da-e41f-4485-b668-c8db12fba3b9 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:54.604523 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b195ac-fd44-4473-a78d-2833cc16e857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:54.605172 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b195ac-fd44-4473-a78d-2833cc16e857 admin admin] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:54.605455 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b195ac-fd44-4473-a78d-2833cc16e857 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:54.605737 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b195ac-fd44-4473-a78d-2833cc16e857 admin admin] PATH_INFO: `/v3/roles` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:54.606453 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b195ac-fd44-4473-a78d-2833cc16e857 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:54.608050 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b195ac-fd44-4473-a78d-2833cc16e857 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:54.618737 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1448/2855] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:52:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:52:54.622638 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3c3c67-6d38-4503-bf03-966a36c2cc51 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:54.633169 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3c3c67-6d38-4503-bf03-966a36c2cc51 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:54.637752 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3687a8b9-1768-46d1-896b-048ea2de0477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:54.638404 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3687a8b9-1768-46d1-896b-048ea2de0477 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:54.638688 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3687a8b9-1768-46d1-896b-048ea2de0477 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:54.638972 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3687a8b9-1768-46d1-896b-048ea2de0477 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:54.639733 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3687a8b9-1768-46d1-896b-048ea2de0477 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:54.641328 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3687a8b9-1768-46d1-896b-048ea2de0477 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:54.650967 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1408/2856] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:52:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 14:52:54.655116 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b195ac-fd44-4473-a78d-2833cc16e857 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:54.664918 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b195ac-fd44-4473-a78d-2833cc16e857 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:54.669428 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8a20e2-ba8f-46ef-bca0-1767418777bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:54.670306 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a20e2-ba8f-46ef-bca0-1767418777bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:54.670589 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a20e2-ba8f-46ef-bca0-1767418777bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:54.670868 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a20e2-ba8f-46ef-bca0-1767418777bd admin admin] PATH_INFO: `/v3/projects/deac7502c4684bdcb5dd9151d6c09894/users/9e7e40e0dfbf49a58d67b897f4792399/roles/3771341b3be641139e752aab690397a4` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:54.685762 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8a20e2-ba8f-46ef-bca0-1767418777bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=deac7502c4684bdcb5dd9151d6c09894, user_id=9e7e40e0dfbf49a58d67b897f4792399, role_id=3771341b3be641139e752aab690397a4)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:54.688453 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8a20e2-ba8f-46ef-bca0-1767418777bd admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:54.784050 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1449/2857] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:52:54 2022] PUT /identity/v3/projects/deac7502c4684bdcb5dd9151d6c09894/users/9e7e40e0dfbf49a58d67b897f4792399/roles/3771341b3be641139e752aab690397a4 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:52:54.789127 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e1c0fc-1b37-4e1b-b67e-c3de59bfc4db None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:54.789424 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e1c0fc-1b37-4e1b-b67e-c3de59bfc4db None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:54.789709 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e1c0fc-1b37-4e1b-b67e-c3de59bfc4db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:54.809016 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-00e1c0fc-1b37-4e1b-b67e-c3de59bfc4db None None] MFA Rules not processed for user `3bd6d2364d1d4e71977a1f04d01cd108`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:52:54.810414 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-00e1c0fc-1b37-4e1b-b67e-c3de59bfc4db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:52:54.869215 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1409/2858] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:52:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:52:54.878586 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8a20e2-ba8f-46ef-bca0-1767418777bd admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:52:54.880344 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-6e8a20e2-ba8f-46ef-bca0-1767418777bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:52:54.903652 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:52:54.940217 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8a20e2-ba8f-46ef-bca0-1767418777bd admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:52:54.944934 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ece7b0e-1c12-4a70-a840-122abf8e726c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:52:54.948236 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ece7b0e-1c12-4a70-a840-122abf8e726c service neutron] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:54.948236 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ece7b0e-1c12-4a70-a840-122abf8e726c service neutron] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:54.948236 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ece7b0e-1c12-4a70-a840-122abf8e726c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:54.961293 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ece7b0e-1c12-4a70-a840-122abf8e726c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:52:54.962843 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ece7b0e-1c12-4a70-a840-122abf8e726c service neutron] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:52:54.974952 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1450/2859] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:52:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:52:57.214963 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b015e702-7abf-46b6-9a02-95e8fbdaef5f None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:52:57.215621 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b015e702-7abf-46b6-9a02-95e8fbdaef5f None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:52:57.215975 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b015e702-7abf-46b6-9a02-95e8fbdaef5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:52:57.234599 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-b015e702-7abf-46b6-9a02-95e8fbdaef5f None None] MFA Rules not processed for user `d2228e3ceb4c4f94bf4207ee9a8aac98`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:52:57.235932 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-b015e702-7abf-46b6-9a02-95e8fbdaef5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:52:57.305317 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1410/2860] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:52:57 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:53:01.271538 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a73c436-cd61-4bac-878b-4bfe2b66b117 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:53:01.271994 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a73c436-cd61-4bac-878b-4bfe2b66b117 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:53:01.272152 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a73c436-cd61-4bac-878b-4bfe2b66b117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:53:01.304424 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-3a73c436-cd61-4bac-878b-4bfe2b66b117 None None] MFA Rules not processed for user `9e7e40e0dfbf49a58d67b897f4792399`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:53:01.305757 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-3a73c436-cd61-4bac-878b-4bfe2b66b117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:53:01.393414 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1451/2861] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:53:01 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:53:01.399026 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b015e702-7abf-46b6-9a02-95e8fbdaef5f None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:53:01.411270 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b015e702-7abf-46b6-9a02-95e8fbdaef5f None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:53:01.420981 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73624e23-3a10-4dfe-baf3-b2145598091e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:53:01.420981 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73624e23-3a10-4dfe-baf3-b2145598091e admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:53:01.420981 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73624e23-3a10-4dfe-baf3-b2145598091e admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:53:01.420981 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73624e23-3a10-4dfe-baf3-b2145598091e admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:53:01.421443 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73624e23-3a10-4dfe-baf3-b2145598091e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:53:01.421443 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73624e23-3a10-4dfe-baf3-b2145598091e admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:53:01.543098 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1411/2862] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jun 7 14:53:01 2022] POST /identity/v3/projects => generated 371 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 14:53:01.546580 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a73c436-cd61-4bac-878b-4bfe2b66b117 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:53:01.560707 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a73c436-cd61-4bac-878b-4bfe2b66b117 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:53:01.626865 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c99724-65e7-4d2c-adff-d5c07d9726dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:53:01.627549 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c99724-65e7-4d2c-adff-d5c07d9726dd admin admin] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:53:01.627833 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c99724-65e7-4d2c-adff-d5c07d9726dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:53:01.628111 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c99724-65e7-4d2c-adff-d5c07d9726dd admin admin] PATH_INFO: `/v3/users` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:53:01.628857 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c99724-65e7-4d2c-adff-d5c07d9726dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:53:01.631538 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c99724-65e7-4d2c-adff-d5c07d9726dd admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:53:01.900783 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1452/2863] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jun 7 14:53:01 2022] POST /identity/v3/users => generated 292 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 14:53:01.904766 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73624e23-3a10-4dfe-baf3-b2145598091e admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:53:01.912168 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:53:01.916489 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73624e23-3a10-4dfe-baf3-b2145598091e admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:53:01.922358 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feba5b8d-533d-4bd0-8e90-e902334f44eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:53:01.928918 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feba5b8d-533d-4bd0-8e90-e902334f44eb admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:53:01.928918 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feba5b8d-533d-4bd0-8e90-e902334f44eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:53:01.928918 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feba5b8d-533d-4bd0-8e90-e902334f44eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:53:01.928918 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feba5b8d-533d-4bd0-8e90-e902334f44eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:53:01.928918 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feba5b8d-533d-4bd0-8e90-e902334f44eb admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:53:01.945034 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1412/2864] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:53:01 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 14:53:01.949408 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c99724-65e7-4d2c-adff-d5c07d9726dd admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:53:01.964305 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c99724-65e7-4d2c-adff-d5c07d9726dd admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:53:01.967625 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22060134-f7d1-4e7e-ab3d-5ad58ccae6c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:53:01.968571 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22060134-f7d1-4e7e-ab3d-5ad58ccae6c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:53:01.968865 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22060134-f7d1-4e7e-ab3d-5ad58ccae6c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:53:01.969149 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22060134-f7d1-4e7e-ab3d-5ad58ccae6c0 admin admin] PATH_INFO: `/v3/projects/f23802f3c80d4a44869e7bbd72e0e92c/users/44308f431991466cb4a855a443ec5c95/roles/9baff2b18f2649f7b757ff2e1f63c853` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:53:01.986945 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22060134-f7d1-4e7e-ab3d-5ad58ccae6c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f23802f3c80d4a44869e7bbd72e0e92c, user_id=44308f431991466cb4a855a443ec5c95, role_id=9baff2b18f2649f7b757ff2e1f63c853)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:53:01.991958 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22060134-f7d1-4e7e-ab3d-5ad58ccae6c0 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:53:02.072343 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1453/2865] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:53:01 2022] PUT /identity/v3/projects/f23802f3c80d4a44869e7bbd72e0e92c/users/44308f431991466cb4a855a443ec5c95/roles/9baff2b18f2649f7b757ff2e1f63c853 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:53:02.073191 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feba5b8d-533d-4bd0-8e90-e902334f44eb admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:53:02.085977 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feba5b8d-533d-4bd0-8e90-e902334f44eb admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:53:02.172479 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be60f74d-5406-48f6-be04-3739373cbd3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:53:02.174589 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be60f74d-5406-48f6-be04-3739373cbd3e admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:53:02.174589 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be60f74d-5406-48f6-be04-3739373cbd3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:53:02.174589 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be60f74d-5406-48f6-be04-3739373cbd3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:53:02.174865 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be60f74d-5406-48f6-be04-3739373cbd3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:53:02.176293 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be60f74d-5406-48f6-be04-3739373cbd3e admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:53:02.189502 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1413/2866] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jun 7 14:53:02 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 14:53:02.194319 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22060134-f7d1-4e7e-ab3d-5ad58ccae6c0 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:53:02.207445 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22060134-f7d1-4e7e-ab3d-5ad58ccae6c0 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:53:02.212528 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef01f35-1b24-4427-8d03-4ab4d0b76e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:53:02.213425 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef01f35-1b24-4427-8d03-4ab4d0b76e3d admin admin] REQUEST_METHOD: `PUT` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:53:02.213706 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef01f35-1b24-4427-8d03-4ab4d0b76e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:53:02.213981 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef01f35-1b24-4427-8d03-4ab4d0b76e3d admin admin] PATH_INFO: `/v3/projects/f23802f3c80d4a44869e7bbd72e0e92c/users/44308f431991466cb4a855a443ec5c95/roles/9baff2b18f2649f7b757ff2e1f63c853` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:53:02.216029 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef01f35-1b24-4427-8d03-4ab4d0b76e3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f23802f3c80d4a44869e7bbd72e0e92c, user_id=44308f431991466cb4a855a443ec5c95, role_id=9baff2b18f2649f7b757ff2e1f63c853)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:53:02.218734 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef01f35-1b24-4427-8d03-4ab4d0b76e3d admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:53:02.232492 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1454/2867] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jun 7 14:53:02 2022] PUT /identity/v3/projects/f23802f3c80d4a44869e7bbd72e0e92c/users/44308f431991466cb4a855a443ec5c95/roles/9baff2b18f2649f7b757ff2e1f63c853 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:53:08.126304 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0629ef-59b2-4799-8e7e-1d3d8b6c3e54 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:53:08.126795 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0629ef-59b2-4799-8e7e-1d3d8b6c3e54 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:53:08.127006 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0629ef-59b2-4799-8e7e-1d3d8b6c3e54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:53:08.169478 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-cf0629ef-59b2-4799-8e7e-1d3d8b6c3e54 None None] MFA Rules not processed for user `44308f431991466cb4a855a443ec5c95`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:53:08.170769 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-cf0629ef-59b2-4799-8e7e-1d3d8b6c3e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:53:08.256764 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1414/2868] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:53:08 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:53:08.271535 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef01f35-1b24-4427-8d03-4ab4d0b76e3d admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:53:08.273377 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-3ef01f35-1b24-4427-8d03-4ab4d0b76e3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:53:08.280036 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:53:08.428075 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef01f35-1b24-4427-8d03-4ab4d0b76e3d admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:53:08.440570 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e4b2af-1824-49fb-8320-52e0453c5206 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e857db4902e42989d0d040bbc8517c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:53:08.441400 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e4b2af-1824-49fb-8320-52e0453c5206 service nova] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:53:08.441686 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e4b2af-1824-49fb-8320-52e0453c5206 service nova] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:53:08.442041 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e4b2af-1824-49fb-8320-52e0453c5206 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:53:08.507744 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e4b2af-1824-49fb-8320-52e0453c5206 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:53:08.509399 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e4b2af-1824-49fb-8320-52e0453c5206 service nova] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:53:08.525963 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1455/2869] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:53:08 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:53:11.426114 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f31083-c931-4dda-9a45-820a684204ca None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:53:11.429400 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f31083-c931-4dda-9a45-820a684204ca None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:53:11.429400 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f31083-c931-4dda-9a45-820a684204ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:53:11.445538 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-a7f31083-c931-4dda-9a45-820a684204ca None None] MFA Rules not processed for user `9e7e40e0dfbf49a58d67b897f4792399`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:53:11.446771 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-a7f31083-c931-4dda-9a45-820a684204ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:53:11.453743 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1415/2870] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jun 7 14:53:11 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:53:18.639927 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=66 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:53:21.233473 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e4b2af-1824-49fb-8320-52e0453c5206 service nova] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:53:21.235305 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-00e4b2af-1824-49fb-8320-52e0453c5206 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:53:21.254066 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e4b2af-1824-49fb-8320-52e0453c5206 service nova] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:53:21.260193 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cb8ec4-45f0-4ae7-8f4d-9f936c0740d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e857db4902e42989d0d040bbc8517c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:53:21.260911 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cb8ec4-45f0-4ae7-8f4d-9f936c0740d0 service nova] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:53:21.261200 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cb8ec4-45f0-4ae7-8f4d-9f936c0740d0 service nova] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:53:21.261559 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cb8ec4-45f0-4ae7-8f4d-9f936c0740d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:53:21.263692 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-b6cb8ec4-45f0-4ae7-8f4d-9f936c0740d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:53:21.279469 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cb8ec4-45f0-4ae7-8f4d-9f936c0740d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:53:21.280791 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cb8ec4-45f0-4ae7-8f4d-9f936c0740d0 service nova] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:53:21.299651 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1456/2871] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:53:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:53:21.439630 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=69 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:53:28.391340 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b81996-96f6-4e8d-9c47-daf2b69c0f56 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:53:28.391959 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b81996-96f6-4e8d-9c47-daf2b69c0f56 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:53:28.391959 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b81996-96f6-4e8d-9c47-daf2b69c0f56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:53:28.409460 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-89b81996-96f6-4e8d-9c47-daf2b69c0f56 None None] MFA Rules not processed for user `ad9d021b171244d796c320aa9f8450a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:53:28.410635 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-89b81996-96f6-4e8d-9c47-daf2b69c0f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:53:28.468589 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1416/2872] 10.222.0.85 () {58 vars in 1009 bytes} [Tue Jun 7 14:53:28 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:53:28.623351 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cb8ec4-45f0-4ae7-8f4d-9f936c0740d0 service nova] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:53:28.624926 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-b6cb8ec4-45f0-4ae7-8f4d-9f936c0740d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:53:28.697798 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cb8ec4-45f0-4ae7-8f4d-9f936c0740d0 service nova] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:53:28.702813 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981aa0de-a03a-47f4-a46a-f446b598d257 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35aa67025f66459da0c0fc3cbfd45430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:53:28.703465 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981aa0de-a03a-47f4-a46a-f446b598d257 service swift] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:53:28.703749 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981aa0de-a03a-47f4-a46a-f446b598d257 service swift] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:53:28.704022 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981aa0de-a03a-47f4-a46a-f446b598d257 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:53:28.713513 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981aa0de-a03a-47f4-a46a-f446b598d257 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:53:28.715102 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981aa0de-a03a-47f4-a46a-f446b598d257 service swift] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:53:28.723335 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1457/2873] 10.222.0.85 () {60 vars in 1384 bytes} [Tue Jun 7 14:53:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 14:53:38.461156 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=25 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:53:38.723633 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=38 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:53:48.029371 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4118581f-e27b-490a-adfa-192a25b495b1 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:53:48.029830 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4118581f-e27b-490a-adfa-192a25b495b1 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:53:48.029921 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4118581f-e27b-490a-adfa-192a25b495b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:53:48.049583 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-4118581f-e27b-490a-adfa-192a25b495b1 None None] MFA Rules not processed for user `ad9d021b171244d796c320aa9f8450a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:53:48.050859 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-4118581f-e27b-490a-adfa-192a25b495b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:53:48.059273 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1417/2874] 10.222.0.85 () {58 vars in 967 bytes} [Tue Jun 7 14:53:48 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:53:48.072389 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981aa0de-a03a-47f4-a46a-f446b598d257 service swift] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:53:48.084171 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981aa0de-a03a-47f4-a46a-f446b598d257 service swift] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:53:48.089625 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcba8277-06c0-4cff-853e-201a866c40dd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35aa67025f66459da0c0fc3cbfd45430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:53:48.089988 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcba8277-06c0-4cff-853e-201a866c40dd service swift] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:53:48.090271 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcba8277-06c0-4cff-853e-201a866c40dd service swift] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:53:48.090546 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcba8277-06c0-4cff-853e-201a866c40dd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:53:48.100766 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcba8277-06c0-4cff-853e-201a866c40dd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:53:48.102683 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcba8277-06c0-4cff-853e-201a866c40dd service swift] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:53:48.113078 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1458/2875] 10.222.0.85 () {60 vars in 1384 bytes} [Tue Jun 7 14:53:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 07 14:53:50.124460 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4118581f-e27b-490a-adfa-192a25b495b1 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:53:50.181327 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4118581f-e27b-490a-adfa-192a25b495b1 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:53:50.185069 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3977ad1-2337-4209-99f4-f9ec3f045e4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:53:50.185719 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3977ad1-2337-4209-99f4-f9ec3f045e4c service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:53:50.186001 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3977ad1-2337-4209-99f4-f9ec3f045e4c service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:53:50.186276 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3977ad1-2337-4209-99f4-f9ec3f045e4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:53:50.188190 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-a3977ad1-2337-4209-99f4-f9ec3f045e4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:53:50.259653 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3977ad1-2337-4209-99f4-f9ec3f045e4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:53:50.261370 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3977ad1-2337-4209-99f4-f9ec3f045e4c service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:53:50.272614 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1418/2876] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:53:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:53:50.978791 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcba8277-06c0-4cff-853e-201a866c40dd service swift] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:53:50.994818 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcba8277-06c0-4cff-853e-201a866c40dd service swift] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:53:51.001095 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d85c5b7-2c0c-408e-bc31-1ecbd3d76034 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e857db4902e42989d0d040bbc8517c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:53:51.001754 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d85c5b7-2c0c-408e-bc31-1ecbd3d76034 service nova] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:53:51.002227 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d85c5b7-2c0c-408e-bc31-1ecbd3d76034 service nova] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:53:51.002824 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d85c5b7-2c0c-408e-bc31-1ecbd3d76034 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:53:51.005106 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-5d85c5b7-2c0c-408e-bc31-1ecbd3d76034 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:53:51.079076 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3977ad1-2337-4209-99f4-f9ec3f045e4c service neutron] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:53:51.080534 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-a3977ad1-2337-4209-99f4-f9ec3f045e4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:53:51.090091 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d85c5b7-2c0c-408e-bc31-1ecbd3d76034 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:53:51.091715 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d85c5b7-2c0c-408e-bc31-1ecbd3d76034 service nova] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:53:51.102774 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1459/2877] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:53:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:53:51.167991 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3977ad1-2337-4209-99f4-f9ec3f045e4c service neutron] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:53:51.174018 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c55bb7-95d4-4dd0-ae74-7b91710f1b69 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6330294b1c8e4dcd85aad189f007b0aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:53:51.174203 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c55bb7-95d4-4dd0-ae74-7b91710f1b69 service placement] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:53:51.174368 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c55bb7-95d4-4dd0-ae74-7b91710f1b69 service placement] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:53:51.174652 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c55bb7-95d4-4dd0-ae74-7b91710f1b69 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:53:51.176472 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-51c55bb7-95d4-4dd0-ae74-7b91710f1b69 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:53:51.189173 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c55bb7-95d4-4dd0-ae74-7b91710f1b69 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:53:51.190783 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c55bb7-95d4-4dd0-ae74-7b91710f1b69 service placement] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:53:51.202378 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1419/2878] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:53:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:53:51.216873 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d85c5b7-2c0c-408e-bc31-1ecbd3d76034 service nova] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:53:51.219355 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-5d85c5b7-2c0c-408e-bc31-1ecbd3d76034 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:53:51.234776 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d85c5b7-2c0c-408e-bc31-1ecbd3d76034 service nova] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:53:51.241222 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f24d82-fb4c-4d08-bfe1-856fa7e69d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:53:51.242091 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f24d82-fb4c-4d08-bfe1-856fa7e69d49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:53:51.242486 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f24d82-fb4c-4d08-bfe1-856fa7e69d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:53:51.242873 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f24d82-fb4c-4d08-bfe1-856fa7e69d49 admin admin] PATH_INFO: `/v3/users/d2228e3ceb4c4f94bf4207ee9a8aac98` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:53:51.244522 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f24d82-fb4c-4d08-bfe1-856fa7e69d49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2228e3ceb4c4f94bf4207ee9a8aac98)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:53:51.247141 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f24d82-fb4c-4d08-bfe1-856fa7e69d49 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:53:51.480863 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-93f24d82-fb4c-4d08-bfe1-856fa7e69d49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2228e3ceb4c4f94bf4207ee9a8aac98'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:53:51.488005 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-93f24d82-fb4c-4d08-bfe1-856fa7e69d49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2228e3ceb4c4f94bf4207ee9a8aac98'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:53:51.488005 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-93f24d82-fb4c-4d08-bfe1-856fa7e69d49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2228e3ceb4c4f94bf4207ee9a8aac98'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:53:51.493906 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-93f24d82-fb4c-4d08-bfe1-856fa7e69d49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2228e3ceb4c4f94bf4207ee9a8aac98'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:53:51.506620 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-93f24d82-fb4c-4d08-bfe1-856fa7e69d49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2228e3ceb4c4f94bf4207ee9a8aac98'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:53:51.577074 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1460/2879] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:53:51 2022] DELETE /identity/v3/users/d2228e3ceb4c4f94bf4207ee9a8aac98 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:53:52.105378 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c55bb7-95d4-4dd0-ae74-7b91710f1b69 service placement] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:53:52.120707 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c55bb7-95d4-4dd0-ae74-7b91710f1b69 service placement] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:53:52.182139 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:53:52.230918 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ff0829-b7d6-47f4-8306-cbb26c507b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:53:52.231820 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ff0829-b7d6-47f4-8306-cbb26c507b23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:53:52.233467 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ff0829-b7d6-47f4-8306-cbb26c507b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:53:52.233951 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ff0829-b7d6-47f4-8306-cbb26c507b23 admin admin] PATH_INFO: `/v3/projects/005732c774f145ffadd2a7bcb3049f26` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:53:52.235646 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ff0829-b7d6-47f4-8306-cbb26c507b23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=005732c774f145ffadd2a7bcb3049f26)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:53:52.238670 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ff0829-b7d6-47f4-8306-cbb26c507b23 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:53:52.425727 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a5ff0829-b7d6-47f4-8306-cbb26c507b23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '005732c774f145ffadd2a7bcb3049f26', 'request_id': 'req-a5ff0829-b7d6-47f4-8306-cbb26c507b23'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:53:52.545856 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a5ff0829-b7d6-47f4-8306-cbb26c507b23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '005732c774f145ffadd2a7bcb3049f26', 'request_id': 'req-a5ff0829-b7d6-47f4-8306-cbb26c507b23'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:53:52.554311 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a5ff0829-b7d6-47f4-8306-cbb26c507b23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '005732c774f145ffadd2a7bcb3049f26', 'request_id': 'req-a5ff0829-b7d6-47f4-8306-cbb26c507b23'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:53:52.575955 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a5ff0829-b7d6-47f4-8306-cbb26c507b23 admin admin] The token cache is being invalidate because project 005732c774f145ffadd2a7bcb3049f26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:53:52.584543 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a5ff0829-b7d6-47f4-8306-cbb26c507b23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:53:52.584543 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a5ff0829-b7d6-47f4-8306-cbb26c507b23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:53:52.584543 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1420/2880] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:53:52 2022] DELETE /identity/v3/projects/005732c774f145ffadd2a7bcb3049f26 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:53:59.096707 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f24d82-fb4c-4d08-bfe1-856fa7e69d49 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:53:59.098185 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-93f24d82-fb4c-4d08-bfe1-856fa7e69d49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:53:59.112676 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f24d82-fb4c-4d08-bfe1-856fa7e69d49 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:53:59.164511 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:53:59.170584 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53e3f4d-c1e7-494d-a85e-c95e58319731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:53:59.171469 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53e3f4d-c1e7-494d-a85e-c95e58319731 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:53:59.171753 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53e3f4d-c1e7-494d-a85e-c95e58319731 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:53:59.175945 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53e3f4d-c1e7-494d-a85e-c95e58319731 admin admin] PATH_INFO: `/v3/users/0a15f5fa82414bdf8ca3f6723c0a8d2d` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:53:59.187004 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53e3f4d-c1e7-494d-a85e-c95e58319731 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a15f5fa82414bdf8ca3f6723c0a8d2d)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:53:59.189022 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53e3f4d-c1e7-494d-a85e-c95e58319731 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:53:59.435812 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-e53e3f4d-c1e7-494d-a85e-c95e58319731 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a15f5fa82414bdf8ca3f6723c0a8d2d'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:53:59.436500 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-e53e3f4d-c1e7-494d-a85e-c95e58319731 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a15f5fa82414bdf8ca3f6723c0a8d2d'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:53:59.437132 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-e53e3f4d-c1e7-494d-a85e-c95e58319731 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a15f5fa82414bdf8ca3f6723c0a8d2d'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:53:59.446761 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-e53e3f4d-c1e7-494d-a85e-c95e58319731 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a15f5fa82414bdf8ca3f6723c0a8d2d'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:53:59.462469 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-e53e3f4d-c1e7-494d-a85e-c95e58319731 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a15f5fa82414bdf8ca3f6723c0a8d2d'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:53:59.564790 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1461/2881] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:53:59 2022] DELETE /identity/v3/users/0a15f5fa82414bdf8ca3f6723c0a8d2d => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:53:59.838682 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ff0829-b7d6-47f4-8306-cbb26c507b23 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:53:59.840435 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-a5ff0829-b7d6-47f4-8306-cbb26c507b23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:53:59.858133 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ff0829-b7d6-47f4-8306-cbb26c507b23 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:53:59.915711 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401d8b26-13a5-4290-a935-d72855c65a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:53:59.916588 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401d8b26-13a5-4290-a935-d72855c65a8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:53:59.916872 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401d8b26-13a5-4290-a935-d72855c65a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:53:59.917147 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401d8b26-13a5-4290-a935-d72855c65a8e admin admin] PATH_INFO: `/v3/projects/93e877695d0f46339cb4b9b86aac3662` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:53:59.929444 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401d8b26-13a5-4290-a935-d72855c65a8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=93e877695d0f46339cb4b9b86aac3662)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:53:59.931395 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401d8b26-13a5-4290-a935-d72855c65a8e admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:54:00.041631 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-401d8b26-13a5-4290-a935-d72855c65a8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93e877695d0f46339cb4b9b86aac3662', 'request_id': 'req-401d8b26-13a5-4290-a935-d72855c65a8e'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:54:00.129228 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-401d8b26-13a5-4290-a935-d72855c65a8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93e877695d0f46339cb4b9b86aac3662', 'request_id': 'req-401d8b26-13a5-4290-a935-d72855c65a8e'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:54:00.132283 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:54:00.136856 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-401d8b26-13a5-4290-a935-d72855c65a8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93e877695d0f46339cb4b9b86aac3662', 'request_id': 'req-401d8b26-13a5-4290-a935-d72855c65a8e'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:54:00.150824 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-401d8b26-13a5-4290-a935-d72855c65a8e admin admin] The token cache is being invalidate because project 93e877695d0f46339cb4b9b86aac3662 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:54:00.151115 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-401d8b26-13a5-4290-a935-d72855c65a8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:54:00.151914 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-401d8b26-13a5-4290-a935-d72855c65a8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:54:00.153861 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1421/2882] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:53:59 2022] DELETE /identity/v3/projects/93e877695d0f46339cb4b9b86aac3662 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:54:00.157775 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53e3f4d-c1e7-494d-a85e-c95e58319731 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:54:00.159282 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-e53e3f4d-c1e7-494d-a85e-c95e58319731 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:54:00.175004 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53e3f4d-c1e7-494d-a85e-c95e58319731 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:54:00.240689 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e95b87a-e95f-46cf-8b5b-c7b12df351a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:54:00.241595 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e95b87a-e95f-46cf-8b5b-c7b12df351a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:54:00.241880 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e95b87a-e95f-46cf-8b5b-c7b12df351a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:54:00.242160 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e95b87a-e95f-46cf-8b5b-c7b12df351a4 admin admin] PATH_INFO: `/v3/users/0d3a7b29c42549c1929e3dcf96b09b1e` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:54:00.255225 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e95b87a-e95f-46cf-8b5b-c7b12df351a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d3a7b29c42549c1929e3dcf96b09b1e)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:54:00.257198 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e95b87a-e95f-46cf-8b5b-c7b12df351a4 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:54:00.443788 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-0e95b87a-e95f-46cf-8b5b-c7b12df351a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d3a7b29c42549c1929e3dcf96b09b1e'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:54:00.444503 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-0e95b87a-e95f-46cf-8b5b-c7b12df351a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d3a7b29c42549c1929e3dcf96b09b1e'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:54:00.445125 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-0e95b87a-e95f-46cf-8b5b-c7b12df351a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d3a7b29c42549c1929e3dcf96b09b1e'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:54:00.453693 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-0e95b87a-e95f-46cf-8b5b-c7b12df351a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d3a7b29c42549c1929e3dcf96b09b1e'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:54:00.455993 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:54:00.474159 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-0e95b87a-e95f-46cf-8b5b-c7b12df351a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d3a7b29c42549c1929e3dcf96b09b1e'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:54:00.548826 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1462/2883] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:54:00 2022] DELETE /identity/v3/users/0d3a7b29c42549c1929e3dcf96b09b1e => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:54:00.848979 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401d8b26-13a5-4290-a935-d72855c65a8e admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:54:00.850459 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-401d8b26-13a5-4290-a935-d72855c65a8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:54:00.880729 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401d8b26-13a5-4290-a935-d72855c65a8e admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:54:00.886723 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e95b87a-e95f-46cf-8b5b-c7b12df351a4 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:54:00.887210 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-0e95b87a-e95f-46cf-8b5b-c7b12df351a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:54:00.952453 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e95b87a-e95f-46cf-8b5b-c7b12df351a4 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:54:00.952875 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8b8b6a-f721-4fb3-9811-eb24994a4e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:54:00.953755 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b8b6a-f721-4fb3-9811-eb24994a4e4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:54:00.956306 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcaefef-0899-47cb-a8a4-864ea939d81d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:54:00.957042 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcaefef-0899-47cb-a8a4-864ea939d81d service neutron] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:54:00.957328 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcaefef-0899-47cb-a8a4-864ea939d81d service neutron] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:54:00.957819 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcaefef-0899-47cb-a8a4-864ea939d81d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:54:00.959486 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-8bcaefef-0899-47cb-a8a4-864ea939d81d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:54:00.961575 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b8b6a-f721-4fb3-9811-eb24994a4e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:54:00.961909 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b8b6a-f721-4fb3-9811-eb24994a4e4e admin admin] PATH_INFO: `/v3/projects/8ec9881abe61493b9758792c827b78bb` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:54:00.974353 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8b8b6a-f721-4fb3-9811-eb24994a4e4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ec9881abe61493b9758792c827b78bb)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:54:00.976295 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8b8b6a-f721-4fb3-9811-eb24994a4e4e admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:54:01.026277 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcaefef-0899-47cb-a8a4-864ea939d81d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:54:01.027729 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcaefef-0899-47cb-a8a4-864ea939d81d service neutron] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:54:01.038475 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1463/2884] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:54:00 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 14:54:01.086796 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-0a8b8b6a-f721-4fb3-9811-eb24994a4e4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ec9881abe61493b9758792c827b78bb', 'request_id': 'req-0a8b8b6a-f721-4fb3-9811-eb24994a4e4e'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:54:01.162118 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-0a8b8b6a-f721-4fb3-9811-eb24994a4e4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ec9881abe61493b9758792c827b78bb', 'request_id': 'req-0a8b8b6a-f721-4fb3-9811-eb24994a4e4e'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:54:01.170731 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-0a8b8b6a-f721-4fb3-9811-eb24994a4e4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ec9881abe61493b9758792c827b78bb', 'request_id': 'req-0a8b8b6a-f721-4fb3-9811-eb24994a4e4e'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:54:01.187532 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-0a8b8b6a-f721-4fb3-9811-eb24994a4e4e admin admin] The token cache is being invalidate because project 8ec9881abe61493b9758792c827b78bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:54:01.187683 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-0a8b8b6a-f721-4fb3-9811-eb24994a4e4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:54:01.188331 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-0a8b8b6a-f721-4fb3-9811-eb24994a4e4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:54:01.190301 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1422/2885] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:54:00 2022] DELETE /identity/v3/projects/8ec9881abe61493b9758792c827b78bb => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:54:02.831013 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ad1bb3-9189-4e11-95d3-0238f282c40c None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:54:02.831505 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ad1bb3-9189-4e11-95d3-0238f282c40c None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:54:02.831607 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ad1bb3-9189-4e11-95d3-0238f282c40c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:54:02.852276 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-92ad1bb3-9189-4e11-95d3-0238f282c40c None None] MFA Rules not processed for user `ad9d021b171244d796c320aa9f8450a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:54:02.853736 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-92ad1bb3-9189-4e11-95d3-0238f282c40c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:54:02.913070 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1464/2886] 10.222.0.85 () {58 vars in 967 bytes} [Tue Jun 7 14:54:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:54:02.927380 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8b8b6a-f721-4fb3-9811-eb24994a4e4e admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:54:02.928857 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-0a8b8b6a-f721-4fb3-9811-eb24994a4e4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:54:03.008189 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8b8b6a-f721-4fb3-9811-eb24994a4e4e admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:54:03.013449 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae8158d-4cc3-4347-bacc-e55c20872396 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35aa67025f66459da0c0fc3cbfd45430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:54:03.014065 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae8158d-4cc3-4347-bacc-e55c20872396 service swift] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:54:03.014353 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae8158d-4cc3-4347-bacc-e55c20872396 service swift] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:54:03.014639 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae8158d-4cc3-4347-bacc-e55c20872396 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:54:03.021436 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:54:03.024132 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae8158d-4cc3-4347-bacc-e55c20872396 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:54:03.025783 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae8158d-4cc3-4347-bacc-e55c20872396 service swift] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:54:03.034712 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1423/2887] 10.222.0.85 () {60 vars in 1384 bytes} [Tue Jun 7 14:54:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 14:54:04.767883 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ad1bb3-9189-4e11-95d3-0238f282c40c None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:54:04.769345 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-92ad1bb3-9189-4e11-95d3-0238f282c40c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:54:04.803104 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:54:04.827496 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ad1bb3-9189-4e11-95d3-0238f282c40c None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:54:04.831233 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b227946e-b661-4516-a7b9-7d8aa07c2d37 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:54:04.831879 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b227946e-b661-4516-a7b9-7d8aa07c2d37 service neutron] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:54:04.832161 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b227946e-b661-4516-a7b9-7d8aa07c2d37 service neutron] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:54:04.832471 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b227946e-b661-4516-a7b9-7d8aa07c2d37 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:54:04.834248 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-b227946e-b661-4516-a7b9-7d8aa07c2d37 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:54:04.899823 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b227946e-b661-4516-a7b9-7d8aa07c2d37 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:54:04.901568 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b227946e-b661-4516-a7b9-7d8aa07c2d37 service neutron] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:54:04.912237 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1465/2888] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:54:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:54:12.368685 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae8158d-4cc3-4347-bacc-e55c20872396 service swift] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:54:12.370141 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-1ae8158d-4cc3-4347-bacc-e55c20872396 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:54:12.383955 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae8158d-4cc3-4347-bacc-e55c20872396 service swift] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:54:12.388626 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455b1998-105a-45f1-a50c-9f9d8f6b756b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:54:12.389497 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455b1998-105a-45f1-a50c-9f9d8f6b756b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:54:12.389784 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455b1998-105a-45f1-a50c-9f9d8f6b756b admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:54:12.390062 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455b1998-105a-45f1-a50c-9f9d8f6b756b admin admin] PATH_INFO: `/v3/users/f2b3f5c080034b658cbfb311c0c77b26` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:54:12.391273 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455b1998-105a-45f1-a50c-9f9d8f6b756b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2b3f5c080034b658cbfb311c0c77b26)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:54:12.393259 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455b1998-105a-45f1-a50c-9f9d8f6b756b admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:54:12.587127 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-455b1998-105a-45f1-a50c-9f9d8f6b756b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2b3f5c080034b658cbfb311c0c77b26'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:54:12.587802 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-455b1998-105a-45f1-a50c-9f9d8f6b756b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2b3f5c080034b658cbfb311c0c77b26'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:54:12.588519 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-455b1998-105a-45f1-a50c-9f9d8f6b756b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2b3f5c080034b658cbfb311c0c77b26'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:54:12.595401 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-455b1998-105a-45f1-a50c-9f9d8f6b756b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2b3f5c080034b658cbfb311c0c77b26'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:54:12.608165 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-455b1998-105a-45f1-a50c-9f9d8f6b756b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2b3f5c080034b658cbfb311c0c77b26'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:54:12.686756 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1424/2889] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:54:12 2022] DELETE /identity/v3/users/f2b3f5c080034b658cbfb311c0c77b26 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:54:12.934381 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b227946e-b661-4516-a7b9-7d8aa07c2d37 service neutron] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:54:12.949003 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b227946e-b661-4516-a7b9-7d8aa07c2d37 service neutron] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:54:13.006642 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe54617-913e-46c7-b5d2-f8f63c987f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:54:13.007496 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe54617-913e-46c7-b5d2-f8f63c987f17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:54:13.007775 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe54617-913e-46c7-b5d2-f8f63c987f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:54:13.008046 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe54617-913e-46c7-b5d2-f8f63c987f17 admin admin] PATH_INFO: `/v3/projects/70b63567b3e54cf1a4348fb65b65a040` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:54:13.009245 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe54617-913e-46c7-b5d2-f8f63c987f17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70b63567b3e54cf1a4348fb65b65a040)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:54:13.011198 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe54617-913e-46c7-b5d2-f8f63c987f17 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:54:13.127349 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-cfe54617-913e-46c7-b5d2-f8f63c987f17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70b63567b3e54cf1a4348fb65b65a040', 'request_id': 'req-cfe54617-913e-46c7-b5d2-f8f63c987f17'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:54:13.195530 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-cfe54617-913e-46c7-b5d2-f8f63c987f17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70b63567b3e54cf1a4348fb65b65a040', 'request_id': 'req-cfe54617-913e-46c7-b5d2-f8f63c987f17'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:54:13.201842 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-cfe54617-913e-46c7-b5d2-f8f63c987f17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70b63567b3e54cf1a4348fb65b65a040', 'request_id': 'req-cfe54617-913e-46c7-b5d2-f8f63c987f17'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:54:13.213669 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-cfe54617-913e-46c7-b5d2-f8f63c987f17 admin admin] The token cache is being invalidate because project 70b63567b3e54cf1a4348fb65b65a040 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:54:13.213964 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-cfe54617-913e-46c7-b5d2-f8f63c987f17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:54:13.214607 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-cfe54617-913e-46c7-b5d2-f8f63c987f17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:54:13.216188 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1466/2890] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:54:12 2022] DELETE /identity/v3/projects/70b63567b3e54cf1a4348fb65b65a040 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:54:13.220206 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455b1998-105a-45f1-a50c-9f9d8f6b756b admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:54:13.228434 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-455b1998-105a-45f1-a50c-9f9d8f6b756b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:54:13.235514 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455b1998-105a-45f1-a50c-9f9d8f6b756b admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:54:13.293127 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53154d1b-49f2-4cd3-b0b3-ed7c241cea45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:54:13.294095 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53154d1b-49f2-4cd3-b0b3-ed7c241cea45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:54:13.294386 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53154d1b-49f2-4cd3-b0b3-ed7c241cea45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:54:13.294670 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53154d1b-49f2-4cd3-b0b3-ed7c241cea45 admin admin] PATH_INFO: `/v3/users/9a3a6b2727574aa690f87819ffa52c89` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:54:13.295897 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53154d1b-49f2-4cd3-b0b3-ed7c241cea45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a3a6b2727574aa690f87819ffa52c89)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:54:13.297878 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53154d1b-49f2-4cd3-b0b3-ed7c241cea45 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:54:13.530071 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-53154d1b-49f2-4cd3-b0b3-ed7c241cea45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a3a6b2727574aa690f87819ffa52c89'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:54:13.530763 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-53154d1b-49f2-4cd3-b0b3-ed7c241cea45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a3a6b2727574aa690f87819ffa52c89'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:54:13.531327 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-53154d1b-49f2-4cd3-b0b3-ed7c241cea45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a3a6b2727574aa690f87819ffa52c89'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:54:13.538449 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-53154d1b-49f2-4cd3-b0b3-ed7c241cea45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a3a6b2727574aa690f87819ffa52c89'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:54:13.549545 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:54:13.552267 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-53154d1b-49f2-4cd3-b0b3-ed7c241cea45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a3a6b2727574aa690f87819ffa52c89'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:54:13.675966 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1425/2891] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:54:13 2022] DELETE /identity/v3/users/9a3a6b2727574aa690f87819ffa52c89 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:54:13.926327 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe54617-913e-46c7-b5d2-f8f63c987f17 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:54:13.927759 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-cfe54617-913e-46c7-b5d2-f8f63c987f17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:54:13.943778 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe54617-913e-46c7-b5d2-f8f63c987f17 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:54:13.949149 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:54:14.008250 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1231ea1d-6c51-44ab-8903-4998a4848677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:54:14.009056 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1231ea1d-6c51-44ab-8903-4998a4848677 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:54:14.009345 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1231ea1d-6c51-44ab-8903-4998a4848677 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:54:14.009618 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1231ea1d-6c51-44ab-8903-4998a4848677 admin admin] PATH_INFO: `/v3/projects/4a2f8209eba443bb94046ac2df18413d` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:54:14.010797 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1231ea1d-6c51-44ab-8903-4998a4848677 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a2f8209eba443bb94046ac2df18413d)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:54:14.012796 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1231ea1d-6c51-44ab-8903-4998a4848677 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:54:14.135574 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-1231ea1d-6c51-44ab-8903-4998a4848677 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a2f8209eba443bb94046ac2df18413d', 'request_id': 'req-1231ea1d-6c51-44ab-8903-4998a4848677'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:54:14.210891 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-1231ea1d-6c51-44ab-8903-4998a4848677 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a2f8209eba443bb94046ac2df18413d', 'request_id': 'req-1231ea1d-6c51-44ab-8903-4998a4848677'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:54:14.217303 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-1231ea1d-6c51-44ab-8903-4998a4848677 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a2f8209eba443bb94046ac2df18413d', 'request_id': 'req-1231ea1d-6c51-44ab-8903-4998a4848677'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:54:14.229489 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-1231ea1d-6c51-44ab-8903-4998a4848677 admin admin] The token cache is being invalidate because project 4a2f8209eba443bb94046ac2df18413d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80387) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:54:14.229783 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-1231ea1d-6c51-44ab-8903-4998a4848677 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:54:14.230444 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-1231ea1d-6c51-44ab-8903-4998a4848677 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:54:14.232031 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1467/2892] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:54:13 2022] DELETE /identity/v3/projects/4a2f8209eba443bb94046ac2df18413d => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:54:23.838451 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:DELETE=2,keystone:SELECT=3,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:54:24.226423 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:55:26.110346 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53154d1b-49f2-4cd3-b0b3-ed7c241cea45 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:55:26.111788 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-53154d1b-49f2-4cd3-b0b3-ed7c241cea45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:55:26.169911 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53154d1b-49f2-4cd3-b0b3-ed7c241cea45 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:55:26.218870 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b41bcb7-400a-4544-9616-5c28211b4342 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:55:26.219451 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b41bcb7-400a-4544-9616-5c28211b4342 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:55:26.219740 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b41bcb7-400a-4544-9616-5c28211b4342 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:55:26.220046 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b41bcb7-400a-4544-9616-5c28211b4342 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:55:26.221946 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-9b41bcb7-400a-4544-9616-5c28211b4342 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:55:26.232550 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b41bcb7-400a-4544-9616-5c28211b4342 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:55:26.233985 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b41bcb7-400a-4544-9616-5c28211b4342 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:55:26.242184 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1426/2893] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:55:26 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 14:55:36.238469 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=46 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:56:17.572965 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1231ea1d-6c51-44ab-8903-4998a4848677 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:56:17.574423 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-1231ea1d-6c51-44ab-8903-4998a4848677 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:56:17.586909 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1231ea1d-6c51-44ab-8903-4998a4848677 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:56:17.590481 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd7af85-2df6-4448-be43-4887fd9cda6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:56:17.595377 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd7af85-2df6-4448-be43-4887fd9cda6d service neutron] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:56:17.595377 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd7af85-2df6-4448-be43-4887fd9cda6d service neutron] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:56:17.595377 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd7af85-2df6-4448-be43-4887fd9cda6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:56:17.595377 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-acd7af85-2df6-4448-be43-4887fd9cda6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:56:17.672331 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd7af85-2df6-4448-be43-4887fd9cda6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:56:17.673799 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd7af85-2df6-4448-be43-4887fd9cda6d service neutron] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:56:17.685856 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1468/2894] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:56:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:56:18.867515 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b41bcb7-400a-4544-9616-5c28211b4342 service neutron] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:56:18.869109 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-9b41bcb7-400a-4544-9616-5c28211b4342 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:56:18.880093 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b41bcb7-400a-4544-9616-5c28211b4342 service neutron] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:56:18.883991 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be818c6-7fe4-4682-9b6d-1f2dd257e812 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:56:18.884746 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be818c6-7fe4-4682-9b6d-1f2dd257e812 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:56:18.885049 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be818c6-7fe4-4682-9b6d-1f2dd257e812 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:56:18.885318 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be818c6-7fe4-4682-9b6d-1f2dd257e812 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:56:18.887084 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-9be818c6-7fe4-4682-9b6d-1f2dd257e812 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:56:18.900792 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be818c6-7fe4-4682-9b6d-1f2dd257e812 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:56:18.902204 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be818c6-7fe4-4682-9b6d-1f2dd257e812 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:56:18.914453 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1427/2895] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:56:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:56:27.680538 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=34 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:56:28.908515 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=6 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:56:31.992974 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd7af85-2df6-4448-be43-4887fd9cda6d service neutron] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:56:32.001456 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd7af85-2df6-4448-be43-4887fd9cda6d service neutron] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:56:32.005091 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8920101b-8f47-4c96-b4be-f88ec351b242 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:56:32.005727 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8920101b-8f47-4c96-b4be-f88ec351b242 service neutron] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:56:32.006027 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8920101b-8f47-4c96-b4be-f88ec351b242 service neutron] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:56:32.006849 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8920101b-8f47-4c96-b4be-f88ec351b242 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:56:32.008952 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-8920101b-8f47-4c96-b4be-f88ec351b242 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:56:32.018289 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8920101b-8f47-4c96-b4be-f88ec351b242 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:56:32.019710 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8920101b-8f47-4c96-b4be-f88ec351b242 service neutron] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:56:32.027884 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1469/2896] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:56:31 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:56:32.338242 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c42fc90-c391-45b8-873a-9f5544bea2af None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:56:32.338935 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c42fc90-c391-45b8-873a-9f5544bea2af None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:56:32.338935 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c42fc90-c391-45b8-873a-9f5544bea2af None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:56:32.340182 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1428/2897] 10.222.0.85 () {54 vars in 921 bytes} [Tue Jun 7 14:56:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:56:32.347621 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac17b77-415e-49ce-a0eb-31512b736c05 None None] REQUEST_METHOD: `POST` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:56:32.347897 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac17b77-415e-49ce-a0eb-31512b736c05 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:56:32.348178 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac17b77-415e-49ce-a0eb-31512b736c05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:56:32.395732 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.auth.core [None req-eac17b77-415e-49ce-a0eb-31512b736c05 None None] MFA Rules not processed for user `7ac4c85918c045b793a1a53300c989a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80387) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:56:32.397146 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-eac17b77-415e-49ce-a0eb-31512b736c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:56:32.403699 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1470/2898] 10.222.0.85 () {58 vars in 1018 bytes} [Tue Jun 7 14:56:32 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 14:56:32.416253 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c42fc90-c391-45b8-873a-9f5544bea2af None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:56:32.427127 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c42fc90-c391-45b8-873a-9f5544bea2af None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:56:32.431310 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c715ab5-f479-446b-b177-130e08b149f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:56:32.431946 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c715ab5-f479-446b-b177-130e08b149f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:56:32.432231 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c715ab5-f479-446b-b177-130e08b149f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:56:32.432554 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c715ab5-f479-446b-b177-130e08b149f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:56:32.439452 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c715ab5-f479-446b-b177-130e08b149f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:56:32.440938 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c715ab5-f479-446b-b177-130e08b149f2 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:56:32.448644 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1429/2899] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:56:32 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:56:32.530478 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7569bc1e-d38c-4bdb-9a6e-86a29c0349f2 None None] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:56:32.531996 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7569bc1e-d38c-4bdb-9a6e-86a29c0349f2 None None] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:56:32.531996 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7569bc1e-d38c-4bdb-9a6e-86a29c0349f2 None None] PATH_INFO: `/` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:56:32.532742 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1471/2900] 10.222.0.85 () {54 vars in 921 bytes} [Tue Jun 7 14:56:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 14:56:32.538611 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7c6b02-29c5-4573-afce-e7ddffd41e64 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:56:32.538904 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7c6b02-29c5-4573-afce-e7ddffd41e64 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:56:32.539188 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7c6b02-29c5-4573-afce-e7ddffd41e64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:56:32.574681 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-cd7c6b02-29c5-4573-afce-e7ddffd41e64 None None] MFA Rules not processed for user `8e857db4902e42989d0d040bbc8517c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 14:56:32.575864 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-cd7c6b02-29c5-4573-afce-e7ddffd41e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:56:32.584993 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1430/2901] 10.222.0.85 () {58 vars in 1018 bytes} [Tue Jun 7 14:56:32 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 14:56:32.595113 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7569bc1e-d38c-4bdb-9a6e-86a29c0349f2 None None] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:56:32.607662 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7569bc1e-d38c-4bdb-9a6e-86a29c0349f2 None None] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:56:32.613172 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc91496-d46a-4fde-827f-5ee5cdabc094 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:56:32.613819 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc91496-d46a-4fde-827f-5ee5cdabc094 service neutron] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:56:32.614108 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc91496-d46a-4fde-827f-5ee5cdabc094 service neutron] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:56:32.614388 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc91496-d46a-4fde-827f-5ee5cdabc094 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:56:32.627605 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc91496-d46a-4fde-827f-5ee5cdabc094 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:56:32.627806 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc91496-d46a-4fde-827f-5ee5cdabc094 service neutron] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:56:32.641436 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1472/2902] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:56:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:56:32.705022 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7c6b02-29c5-4573-afce-e7ddffd41e64 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:56:32.706402 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-cd7c6b02-29c5-4573-afce-e7ddffd41e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:56:32.776711 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7c6b02-29c5-4573-afce-e7ddffd41e64 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:56:32.789530 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e99f392-2cba-4e07-ba90-86c239dc52b6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6330294b1c8e4dcd85aad189f007b0aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:56:32.793276 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e99f392-2cba-4e07-ba90-86c239dc52b6 service placement] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:56:32.793276 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e99f392-2cba-4e07-ba90-86c239dc52b6 service placement] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:56:32.793276 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e99f392-2cba-4e07-ba90-86c239dc52b6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:56:32.793276 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-8e99f392-2cba-4e07-ba90-86c239dc52b6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:56:32.804949 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e99f392-2cba-4e07-ba90-86c239dc52b6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:56:32.806544 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e99f392-2cba-4e07-ba90-86c239dc52b6 service placement] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:56:32.816511 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1431/2903] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:56:32 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:56:33.410388 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc91496-d46a-4fde-827f-5ee5cdabc094 service neutron] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:56:33.412909 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-9dc91496-d46a-4fde-827f-5ee5cdabc094 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:56:33.427722 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc91496-d46a-4fde-827f-5ee5cdabc094 service neutron] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:56:33.433908 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f04ba8-cfa7-4ecb-8205-65f3a6771767 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e857db4902e42989d0d040bbc8517c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:56:33.434680 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f04ba8-cfa7-4ecb-8205-65f3a6771767 service nova] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:56:33.434969 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f04ba8-cfa7-4ecb-8205-65f3a6771767 service nova] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:56:33.435270 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f04ba8-cfa7-4ecb-8205-65f3a6771767 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:56:33.437506 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-92f04ba8-cfa7-4ecb-8205-65f3a6771767 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:56:33.450205 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f04ba8-cfa7-4ecb-8205-65f3a6771767 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:56:33.451788 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f04ba8-cfa7-4ecb-8205-65f3a6771767 service nova] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:56:33.464595 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1473/2904] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:56:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:56:42.812199 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=47 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:56:43.459006 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=31 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:57:27.392608 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e99f392-2cba-4e07-ba90-86c239dc52b6 service placement] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:57:27.403189 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e99f392-2cba-4e07-ba90-86c239dc52b6 service placement] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:57:27.408548 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9023c22-7aa4-4ab6-a2a8-d059a22611ae service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6330294b1c8e4dcd85aad189f007b0aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:57:27.409188 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9023c22-7aa4-4ab6-a2a8-d059a22611ae service placement] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:57:27.409475 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9023c22-7aa4-4ab6-a2a8-d059a22611ae service placement] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:57:27.409757 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9023c22-7aa4-4ab6-a2a8-d059a22611ae service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:57:27.411537 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-c9023c22-7aa4-4ab6-a2a8-d059a22611ae service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:57:27.423193 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9023c22-7aa4-4ab6-a2a8-d059a22611ae service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:57:27.424793 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9023c22-7aa4-4ab6-a2a8-d059a22611ae service placement] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:57:27.434761 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1432/2905] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:57:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:57:37.430668 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=6 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:58:16.935493 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f04ba8-cfa7-4ecb-8205-65f3a6771767 service nova] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:58:16.947952 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f04ba8-cfa7-4ecb-8205-65f3a6771767 service nova] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:58:16.951686 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f86130-ca83-4c3c-81a3-cdc7f2fc0a61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:58:16.952329 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f86130-ca83-4c3c-81a3-cdc7f2fc0a61 service neutron] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:58:16.952643 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f86130-ca83-4c3c-81a3-cdc7f2fc0a61 service neutron] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:58:16.952923 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f86130-ca83-4c3c-81a3-cdc7f2fc0a61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:58:16.954721 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-13f86130-ca83-4c3c-81a3-cdc7f2fc0a61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:58:17.014669 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f86130-ca83-4c3c-81a3-cdc7f2fc0a61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:58:17.016110 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f86130-ca83-4c3c-81a3-cdc7f2fc0a61 service neutron] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:58:17.025515 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1474/2906] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:58:16 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:58:20.842096 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9023c22-7aa4-4ab6-a2a8-d059a22611ae service placement] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:58:20.855407 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9023c22-7aa4-4ab6-a2a8-d059a22611ae service placement] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:58:20.861088 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ba4af9-50ec-47d8-b87a-8bf8806fd00f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e857db4902e42989d0d040bbc8517c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:58:20.862815 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ba4af9-50ec-47d8-b87a-8bf8806fd00f service nova] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:58:20.862815 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ba4af9-50ec-47d8-b87a-8bf8806fd00f service nova] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:58:20.862815 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ba4af9-50ec-47d8-b87a-8bf8806fd00f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:58:20.864924 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-23ba4af9-50ec-47d8-b87a-8bf8806fd00f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:58:20.882689 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ba4af9-50ec-47d8-b87a-8bf8806fd00f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:58:20.884304 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ba4af9-50ec-47d8-b87a-8bf8806fd00f service nova] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:58:20.895571 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1433/2907] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:58:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:58:23.257277 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f86130-ca83-4c3c-81a3-cdc7f2fc0a61 service neutron] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:58:23.266326 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f86130-ca83-4c3c-81a3-cdc7f2fc0a61 service neutron] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:58:23.270337 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1147cafb-a072-4626-ac15-7560d7a3e881 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac4c85918c045b793a1a53300c989a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:58:23.270982 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1147cafb-a072-4626-ac15-7560d7a3e881 service neutron] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:58:23.271268 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1147cafb-a072-4626-ac15-7560d7a3e881 service neutron] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:58:23.271582 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1147cafb-a072-4626-ac15-7560d7a3e881 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:58:23.273427 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-1147cafb-a072-4626-ac15-7560d7a3e881 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:58:23.338933 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1147cafb-a072-4626-ac15-7560d7a3e881 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:58:23.340397 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1147cafb-a072-4626-ac15-7560d7a3e881 service neutron] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:58:23.350865 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1475/2908] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:58:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:58:26.879487 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ba4af9-50ec-47d8-b87a-8bf8806fd00f service nova] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:58:26.891062 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ba4af9-50ec-47d8-b87a-8bf8806fd00f service nova] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:58:26.896141 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d06ba1-39b8-47bc-98d0-f1019fbc1de6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6330294b1c8e4dcd85aad189f007b0aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:58:26.898483 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d06ba1-39b8-47bc-98d0-f1019fbc1de6 service placement] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:58:26.898483 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d06ba1-39b8-47bc-98d0-f1019fbc1de6 service placement] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:58:26.898483 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d06ba1-39b8-47bc-98d0-f1019fbc1de6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:58:26.899913 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-f4d06ba1-39b8-47bc-98d0-f1019fbc1de6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:58:26.913089 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d06ba1-39b8-47bc-98d0-f1019fbc1de6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:58:26.914673 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d06ba1-39b8-47bc-98d0-f1019fbc1de6 service placement] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:58:26.926035 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1434/2909] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 14:58:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 14:58:30.403272 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1147cafb-a072-4626-ac15-7560d7a3e881 service neutron] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:58:30.404885 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-1147cafb-a072-4626-ac15-7560d7a3e881 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:58:30.421766 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1147cafb-a072-4626-ac15-7560d7a3e881 service neutron] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:58:30.427158 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529c11d1-5677-4d15-8f86-50df2e9aaf2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:58:30.428032 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529c11d1-5677-4d15-8f86-50df2e9aaf2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:58:30.428310 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529c11d1-5677-4d15-8f86-50df2e9aaf2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:58:30.428619 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529c11d1-5677-4d15-8f86-50df2e9aaf2b admin admin] PATH_INFO: `/v3/users/9e7e40e0dfbf49a58d67b897f4792399` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:58:30.429833 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529c11d1-5677-4d15-8f86-50df2e9aaf2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e7e40e0dfbf49a58d67b897f4792399)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:58:30.432013 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529c11d1-5677-4d15-8f86-50df2e9aaf2b admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:58:30.587617 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-529c11d1-5677-4d15-8f86-50df2e9aaf2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e7e40e0dfbf49a58d67b897f4792399'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:58:30.588414 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-529c11d1-5677-4d15-8f86-50df2e9aaf2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e7e40e0dfbf49a58d67b897f4792399'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:58:30.589032 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-529c11d1-5677-4d15-8f86-50df2e9aaf2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e7e40e0dfbf49a58d67b897f4792399'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:58:30.595908 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-529c11d1-5677-4d15-8f86-50df2e9aaf2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e7e40e0dfbf49a58d67b897f4792399'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:58:30.608241 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-529c11d1-5677-4d15-8f86-50df2e9aaf2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e7e40e0dfbf49a58d67b897f4792399'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:58:30.692587 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1476/2910] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:58:30 2022] DELETE /identity/v3/users/9e7e40e0dfbf49a58d67b897f4792399 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:58:30.942464 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d06ba1-39b8-47bc-98d0-f1019fbc1de6 service placement] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:58:30.962726 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d06ba1-39b8-47bc-98d0-f1019fbc1de6 service placement] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:58:31.023112 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06270fa-c9a1-4979-ba9e-5d8fd5c6441c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:58:31.023979 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06270fa-c9a1-4979-ba9e-5d8fd5c6441c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:58:31.024264 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06270fa-c9a1-4979-ba9e-5d8fd5c6441c admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:58:31.024572 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06270fa-c9a1-4979-ba9e-5d8fd5c6441c admin admin] PATH_INFO: `/v3/projects/deac7502c4684bdcb5dd9151d6c09894` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:58:31.025797 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06270fa-c9a1-4979-ba9e-5d8fd5c6441c admin admin] RBAC: Authorizing `identity:delete_project(project_id=deac7502c4684bdcb5dd9151d6c09894)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:58:31.027713 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06270fa-c9a1-4979-ba9e-5d8fd5c6441c admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:58:31.145922 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a06270fa-c9a1-4979-ba9e-5d8fd5c6441c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'deac7502c4684bdcb5dd9151d6c09894', 'request_id': 'req-a06270fa-c9a1-4979-ba9e-5d8fd5c6441c'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:58:31.213506 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a06270fa-c9a1-4979-ba9e-5d8fd5c6441c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'deac7502c4684bdcb5dd9151d6c09894', 'request_id': 'req-a06270fa-c9a1-4979-ba9e-5d8fd5c6441c'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:58:31.219555 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a06270fa-c9a1-4979-ba9e-5d8fd5c6441c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'deac7502c4684bdcb5dd9151d6c09894', 'request_id': 'req-a06270fa-c9a1-4979-ba9e-5d8fd5c6441c'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:58:31.230670 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a06270fa-c9a1-4979-ba9e-5d8fd5c6441c admin admin] The token cache is being invalidate because project deac7502c4684bdcb5dd9151d6c09894 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:58:31.230955 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a06270fa-c9a1-4979-ba9e-5d8fd5c6441c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:58:31.231608 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a06270fa-c9a1-4979-ba9e-5d8fd5c6441c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:58:31.233207 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1435/2911] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:58:30 2022] DELETE /identity/v3/projects/deac7502c4684bdcb5dd9151d6c09894 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:58:31.241907 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529c11d1-5677-4d15-8f86-50df2e9aaf2b admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:58:31.241907 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-529c11d1-5677-4d15-8f86-50df2e9aaf2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:58:31.248746 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:58:31.253356 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529c11d1-5677-4d15-8f86-50df2e9aaf2b admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:58:31.309765 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28dbd84-3067-4bd0-9661-475b87ada185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:58:31.310682 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28dbd84-3067-4bd0-9661-475b87ada185 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:58:31.311014 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28dbd84-3067-4bd0-9661-475b87ada185 admin admin] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:58:31.311331 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28dbd84-3067-4bd0-9661-475b87ada185 admin admin] PATH_INFO: `/v3/users/44308f431991466cb4a855a443ec5c95` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:58:31.312562 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28dbd84-3067-4bd0-9661-475b87ada185 admin admin] RBAC: Authorizing `identity:delete_user(user_id=44308f431991466cb4a855a443ec5c95)` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:58:31.314490 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28dbd84-3067-4bd0-9661-475b87ada185 admin admin] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:58:31.474217 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-c28dbd84-3067-4bd0-9661-475b87ada185 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44308f431991466cb4a855a443ec5c95'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:58:31.474898 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-c28dbd84-3067-4bd0-9661-475b87ada185 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44308f431991466cb4a855a443ec5c95'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:58:31.475472 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-c28dbd84-3067-4bd0-9661-475b87ada185 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44308f431991466cb4a855a443ec5c95'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:58:31.483788 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-c28dbd84-3067-4bd0-9661-475b87ada185 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44308f431991466cb4a855a443ec5c95'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:58:31.498773 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.notifications [None req-c28dbd84-3067-4bd0-9661-475b87ada185 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44308f431991466cb4a855a443ec5c95'} {{(pid=80387) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:58:31.562297 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1477/2912] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jun 7 14:58:31 2022] DELETE /identity/v3/users/44308f431991466cb4a855a443ec5c95 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 14:58:31.820695 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06270fa-c9a1-4979-ba9e-5d8fd5c6441c admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 14:58:31.822176 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-a06270fa-c9a1-4979-ba9e-5d8fd5c6441c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 14:58:31.836432 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06270fa-c9a1-4979-ba9e-5d8fd5c6441c admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 14:58:31.900953 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b4a9df-2e4c-4a88-b852-0863a5e05c24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd6d2364d1d4e71977a1f04d01cd108', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61703df7c02543d6ba1f949d9811ee43', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 14:58:31.901849 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b4a9df-2e4c-4a88-b852-0863a5e05c24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 14:58:31.902134 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b4a9df-2e4c-4a88-b852-0863a5e05c24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 14:58:31.902410 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b4a9df-2e4c-4a88-b852-0863a5e05c24 admin admin] PATH_INFO: `/v3/projects/f23802f3c80d4a44869e7bbd72e0e92c` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 14:58:31.903595 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b4a9df-2e4c-4a88-b852-0863a5e05c24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f23802f3c80d4a44869e7bbd72e0e92c)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 14:58:31.905584 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b4a9df-2e4c-4a88-b852-0863a5e05c24 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 14:58:31.912788 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:58:32.025389 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-c5b4a9df-2e4c-4a88-b852-0863a5e05c24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f23802f3c80d4a44869e7bbd72e0e92c', 'request_id': 'req-c5b4a9df-2e4c-4a88-b852-0863a5e05c24'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:58:32.082462 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-c5b4a9df-2e4c-4a88-b852-0863a5e05c24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f23802f3c80d4a44869e7bbd72e0e92c', 'request_id': 'req-c5b4a9df-2e4c-4a88-b852-0863a5e05c24'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:58:32.088235 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-c5b4a9df-2e4c-4a88-b852-0863a5e05c24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f23802f3c80d4a44869e7bbd72e0e92c', 'request_id': 'req-c5b4a9df-2e4c-4a88-b852-0863a5e05c24'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:58:32.100410 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-c5b4a9df-2e4c-4a88-b852-0863a5e05c24 admin admin] The token cache is being invalidate because project f23802f3c80d4a44869e7bbd72e0e92c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 14:58:32.100695 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-c5b4a9df-2e4c-4a88-b852-0863a5e05c24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:58:32.101347 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-c5b4a9df-2e4c-4a88-b852-0863a5e05c24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 14:58:32.102906 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1436/2913] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jun 7 14:58:31 2022] DELETE /identity/v3/projects/f23802f3c80d4a44869e7bbd72e0e92c => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 14:58:41.558281 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=51,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 14:58:42.096834 dv-cinder-844974-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=22,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 15:02:33.373116 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28dbd84-3067-4bd0-9661-475b87ada185 admin admin] Authenticating user token {{(pid=80387) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 15:02:33.374601 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-c28dbd84-3067-4bd0-9661-475b87ada185 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:02:33.513901 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28dbd84-3067-4bd0-9661-475b87ada185 admin admin] Validating token access rules against request {{(pid=80387) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 15:02:33.574033 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe15f2d-0f14-432b-a0fc-7495e83d5492 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6330294b1c8e4dcd85aad189f007b0aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724785eb74e44daf8736fd01fc0d1839', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80387) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 15:02:33.574687 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe15f2d-0f14-432b-a0fc-7495e83d5492 service placement] REQUEST_METHOD: `GET` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 15:02:33.574976 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe15f2d-0f14-432b-a0fc-7495e83d5492 service placement] SCRIPT_NAME: `/identity` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 15:02:33.575256 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe15f2d-0f14-432b-a0fc-7495e83d5492 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80387) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 15:02:33.577064 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.fernet_utils [None req-abe15f2d-0f14-432b-a0fc-7495e83d5492 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80387) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 15:02:33.589246 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe15f2d-0f14-432b-a0fc-7495e83d5492 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 15:02:33.591271 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe15f2d-0f14-432b-a0fc-7495e83d5492 service placement] RBAC: Authorization granted {{(pid=80387) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 15:02:33.601382 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: [pid: 80387|app: 0|req: 1478/2914] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jun 7 15:02:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 15:02:43.596436 dv-cinder-844974-1-iscsi devstack@keystone.service[80387]: DEBUG dbcounter [-] [80387] Writing DB stats keystone:SELECT=89 {{(pid=80387) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}